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Foreword 


This  report  summarizes  efforts  to  develop  a  rapidly  configurable,  pictorially 
presented,  and  web  administered  measure  of  vocational  interest  for  entry  level  Navy 
enlisted  jobs.  This  is  an  important  component  of  our  research  program  to  overhaul  and 
improve  the  Navy’s  enlisted  selection  and  classification  process.  The  program  is 
designed  to  replace  the  current  classification  algorithm  with  a  more  flexible  and 
accurate  one,  de-emphasize  the  almost  exclusive  focus  on  mental  ability  by  including 
personality  and  interest  measures  in  making  classification  decisions.  Collectively,  these 
efforts  would  transform  and  modernize  enlisted  classification  by  making  it  applicant¬ 
centric  while  improving  job  satisfaction  and  performance,  reducing  attrition,  and 
increasing  continuation  behavior. 

Over  70  years  of  vocational  research  has  demonstrated  that  job  assignments  that  are 
congruent  with  a  person’s  interest  and  preferences,  produce  greater  job  satisfaction, 
increase  performance,  and  lengthen  tenure.  The  difficulty  the  Navy  faces  is  that  enlisted 
applicants  have  limited  work  experience  and  little  knowledge  of  Navy  jobs.  To  overcome 
this,  the  development  focused  on  using  pictures  accompanied  by  small  non-technical 
statements,  items  were  differentiated  based  on  broad  job  characteristics  (e.g.,  industrial 
vs.  office  settings),  and  simple  interest  rating  scales  were  utilized.  The  final  product  is  a 
short  and  engaging  instrument,  based  on  a  model  of  Navy  enlisted  work  that  can  be 
easily  modified.  The  output  from  the  instrument  is  a  rank-ordered  list  of  preferences  for 
all  Navy  entry-level  jobs  which  can  be  used  to  augment  classification  decisions. 

The  research  was  sponsored  by  the  Office  of  Navy  Research  (Code  34)  and  funded 
under  PE  0602236N  and  PE  0603236N. 


DAVID  L.  ALDERTON,  Ph.D. 

Director 
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Executive  Summary 


The  current  investigation  centered  on  the  design,  development,  and  initial  testing  of 
a  computer-administered  pictorial  interest  inventory,  called  Job  Opportunities  in  the 
Navy  (formerly  known  as  Jobs  and  Occupational  Interest  in  the  Navy  [JOIN]).  JOIN 
was  designed  to  measure  recruits’  interest  in  various  facets  of  Navy  occupations. 
Ultimately,  a  validated  version  of  JOIN  would  be  implemented  during  the  classification 
process,  after  selecting  qualified  applicants  largely  based  on  cognitive  abilities. 

The  Navy’s  classification  process  occurs  during  a  relatively  short  period  of  time, 
during  which  the  individual  and  the  organization  must  evaluate  each  other  and  an 
occupational  choice  must  be  made.  During  this  process  individuals  are  offered  a  wide 
variety  of  opportunities  in  a  very  broad  range  of  jobs.  Extensive  vocational  counseling 
literature  attests  to  the  benefits  of  matching  an  incumbent’s  interests  to  job 
requirements.  What  is  more,  the  advantages  may  extend  beyond  those  obtained  from 
job  classification  based  on  pre-existing  abilities  and  knowledge  alone. 

This  study  provided  an  examination  of  a  computer-administered  inventory  that 
centers  on  the  use  of  pictures  as  individual  interest  items.  Content-valid  and  reliable 
pictorial  items  for  an  interest  instrument  were  developed  that  represent  various  Navy 
work  areas,  work  styles,  work  environments,  and  work  activities.  Furthermore,  the 
descriptive  statistical  analyses  of  the  JOIN  items  indicated  that  there  was  adequate 
dispersion  of  individual  responses.  In  other  words,  the  participants  were  different  in 
their  level  of  interest  in  various  items.  Finally,  the  results  from  the  analysis  of  feedback 
items  and  discussion  sessions  suggest  that  incoming  recruits  possess  the  level  of 
computer  proficiency  necessary  to  complete  the  inventory  in  a  limited  amount  of  time. 

In  addition,  recruits  were  typically  positive  in  their  evaluation  of  the  quality  of  the 
computer-adapted  inventory.  Initial  results  suggest  that  recruits  found  the  JOIN 
software  to  be  a  user-friendly  interface. 

The  JOIN  prototype  software  is  in  its  initial  phase  of  development,  and,  as  such,  the 
functionality  of  subsequent  versions  is  potentially  limitless.  Future  testing,  however,  will 
be  conducted  with  a  larger  sample  of  new  recruits  to  determine  the  inventory’s  criterion- 
related  validity.  Participants’  interest  responses  will  be  analyzed  and  matched  with 
relevant  outcome  data,  such  as  attrition  from  training  school  and  future  job  satisfaction. 
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I  ntroduction 


To  meet  their  business  purpose,  most  organizations  must  dedicate  extensive  time 
and  effort  to  acquire  and  maintain  a  high-performance  workforce.  As  the  largest 
employers  in  America,  the  Armed  Forces  are  certainly  no  exception  to  this  requirement. 
A  critical  task  for  the  Armed  Forces  is  the  selection  of  the  appropriate  personnel  and 
their  appropriate  classification  (i.e.,  job  assignment)  during  initial  processing  (Schratz  & 
Ree,  1989).  Because  of  its  role  in  our  society,  successful  personnel  selection  and 
classification  could  be  considered  to  be  more  important  for  the  military  than  for  any 
other  organization,  public  or  private. 

With  respect  to  the  Navy,  its  principal  goal  is  to  maintain  a  viable  defense 
organization  consisting  of  hundreds  of  thousands  of  individuals  who  are  well-qualified 
and  well -trained  (Kroeker,  1989).  Every  year  thousands  of  young  people,  who  have  little 
prior  training,  volunteer  for  the  Navy,  and  for  many  it  is  their  first  exposure  to  the  world 
of  work  (Kroeker,  1989).  In  this  context,  effective  personnel  selection  and  classification 
is  much  more  difficult  in  the  military  than  in  other  organizations.  Selection  occurs  when 
an  organization  chooses  to  accept  or  reject  an  applicant  for  eventual  assignment  to  a  job 
(Gatewood  &  Feild,  1998).  This  decision  is  made  under  legal  and  environmental 
constraints,  and  addresses  the  future  interests  of  the  organization.  Information  is 
systematically  collected  from  applicants  to  determine  how  much  job-related  information 
and  ability  to  perform  job  activities  each  possesses.  Once  an  applicant  has  been  selected, 
he  or  she  must  then  be  assigned  to  a  particular  job.  The  classification  of  qualified 
individuals  is  a  much  more  complicated  process  and  must  exhibit  flexibility.  Optimal 
assignment,  or  matching  the  person  to  the  job,  is  the  lofty  aim  and  ultimate  achievement 
of  successful  selection  and  classification  processes.  A  military  organization’s 
accessioning  policy  involves  a  complex  system  of  goals  and  constraints  that  drives  the 
assignment  of  a  particular  recruit  to  a  particular  job  (Wiskoff  &  Rampton,  1989).  The 
number  of  jobs  available,  the  number  of  applicants,  and  the  qualifications  of  applicants 
are  also  key  parameters  in  determining  actual  assignments  (Kroeker,  1989). 

An  optimal  matching  of  persons  and  jobs  is  one  that  cannot  be  improved  upon  by 
any  other  allocation  (Kroeker,  1989).  Although  there  are  many  standards  by  which 
successful  matching  can  be  evaluated,  one  criterion  often  used  is  person-environment 
(P-E)  fit,  also  referred  to  as  person-organization  (P-O)  congruence.  One  form  of  P-E  fit 
is  the  compatibility  of  individuals  with  specific  jobs,  or  person-job  (P-J)  fit.  Empirically, 
job  performance,  employee  satisfaction,  and  retention  are  contingent  upon 
appropriately  matching  personnel  with  their  desired  occupation  (Bretz  &  Judge,  1994; 
Kristof,  1996;  Tranberg,  Slane,  &  Ekeberg,  1993).  Today’s  military  manpower  and 
personnel  managers  require  classification  procedures  that  work  effectively  under  a 
variety  of  conditions  (Kroeker,  1989).  The  closer  military  managers  can  get  to  optimal 
assignment,  the  lower  the  costs  (e.g.,  early  attrition,  reclassification,  etc.)  that  will  be 
associated  with  the  process  (Kristof,  1996;  Schratz  &  Ree,  1989). 

The  Navy  is  cognizant  of  those  factors  that  function  to  challenge  the  successful 
classification  of  qualified  personnel,  and  it  is  attempting  to  meet  the  challenge  in  part  by 
modifying  existing  personnel  accessioning  procedures  and  in  part  by  developing  new 
technological  products  (Baker  &  Ellis,  1989;  Kroeker,  1989).  The  Navy’s  classification 
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process  occurs  during  a  relatively  short  period  of  time,  during  which  the  individual  and 
the  organization  must  evaluate  each  other  and  an  occupational  choice  must  be  made. 
During  this  process  individuals  are  offered  a  wide  variety  of  opportunities  in  a  very 
broad  range  of  jobs  (Kroeker,  1989).  What  further  complicates  this  procedure  is  that 
most  applicants  know  almost  nothing  about  the  Navy  jobs  (i.e.,  referred  to  as  ratings) 
that  are  available.  Not  only  is  the  Navy  faced  with  the  challenge  of  filling  vacancies  with 
the  most  qualified  applicants,  but  it  must  also  decide  which  job  is  best  suited  for  the 
applicant  while  taking  into  consideration  his  or  her  career  preferences. 

Currently,  a  major  impediment  to  successful  assignment  is  the  use  of  recruiting  goals 
to  drive  accessioning  policies  (Navy  Personnel  Research,  Studies,  and  Technology 
Department  [NPRST],  1998).  In  general,  the  process  results  in  a  limited  amount  of 
information  collected  about  an  applicant’s  past  behavior,  personality,  and  interests. 
Navy  researchers  have  proposed  an  agenda  for  defining  manpower  and  personnel 
research  in  the  future,  which  includes  evaluating  the  benefits  of  a  more  “complete” 
personnel  assessment  beyond  that  of  intelligence  alone.  Research  initiatives  include 
examining  methods  to  improve  the  classification  utility  of  existing  aptitude  measures  as 
operational  predictors  (NPRST,  1998).  This  entails  experimenting  with  alternative 
predictors,  such  as  vocational  interests,  personality  measures,  and  biographical  data, 
that  are  not  yet  part  of  the  operational  ability  measures  (i.e.,  Armed  Service  Vocational 
Aptitude  Battery  [ASVAB])  and  incorporating  this  predictor  information  into  the 
classification  process  (Borman,  Hedge,  Ferstl,  &  Kaufman,  2002). 

A  necessary  step  to  carry  out  the  research  agenda  described  above  is  to  develop  these 
alternative  screening  devices  to  aid  in  classification.  The  current  investigation  centers 
on  the  design,  development,  and  initial  testing  of  a  computer-administered  pictorial 
interest  inventory,  called  Job  Opportunities  in  the  Navy  (formerly  known  as  Jobs  and 
Occupational  Interest  in  the  Navy  [JOIN]).  JOIN  was  designed  to  measure  recruits’ 
interest  in  various  facets  of  Navy  occupations.  Ultimately,  a  validated  version  of  JOIN 
would  be  implemented  during  the  classification  process.  After  selecting  qualified 
applicants  largely  based  on  cognitive  abilities,  interest  measurement  would  be  utilized 
to  match  people  along  a  multitude  of  dimensions  into  the  “best  fitting”  Navy  job 
available.  The  challenge  was  to  develop  a  flexible  inventory  that,  by  design,  would  assess 
applicant  preferences  and  interest.  Extensive  vocational  counseling  literature  attests  to 
the  benefits  of  matching  an  incumbent’s  interests  to  job  requirements  (Barak,  2001; 
Betz,  2001;  Dawis,  1991).  What  is  more,  the  advantages  may  extend  beyond  those 
obtained  from  job  classification  based  on  pre-existing  abilities  and  knowledge  alone 
(Borman  et  al.,  2002).  There  is  evidence  to  suggest  that  when  a  person  is  intellectually 
qualified  for  a  host  of  jobs,  the  best  assignment  for  long-term  retention  is  the  one  that 
fits  an  individual’s  interests  and  not  just  the  organization’s  manpower  needs  (Fitzgerald 
&  Rounds,  1989;  Hunter  &  Hunter,  1984;  Karren  &  Graves,  1994;  Savickas  &  Spokane, 
1999). 

The  current  project  attempts  to  have  its  initial  impact  at  the  decision  point  where 
career  counselors  (i.e.,  classifiers)  and  applicants  discuss  job  options  and  a  job 
assignment  is  made.  The  ultimate  goal  is  to  demonstrate  a  difference  between  the 
quality  of  assignments  actually  made  and  potential  improvements  in  assignments  or  the 
assignment  process.  In  the  future,  this  may  be  accomplished  by  examining  various  P-0 
fit  indicators,  such  as  recruits’  satisfaction  with  the  assignment  process,  training  school 
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performance,  attrition  from  training  school,  and  reclassifications.  The  potential  to 
account  for  incremental  variance  in  these  criteria,  over  that  accounted  for  by  ability 
measures  alone,  served  as  the  impetus  for  this  research  project. 


Person- Organization  (P-O)  Congruence 

Person-Organization  (P-O)  congruence  concerns  the  antecedents  and  consequences 
of  compatibility  between  people  and  the  organizations  in  which  they  work  (Kristof, 
1996).  Congruence  theory  has  dealt  with  broad  issues  of  how  the  characteristics  of  the 
person  and  the  organization’s  environment  interact  to  affect  well-being  (Kulik,  Oldham, 
&  Hackman,  1987).  Generally,  conditions  of  fit  between  the  person  and  the  environment 
are  predicted  to  result  in  high  performance,  satisfaction,  and  low  stress  (Borman  et  al., 
2002;  Karren  &  Graves,  1994;  Kristof,  1996;  Kulik  et  al.,  1987).  Researchers  in  this 
literature  area  often  describe  two  broad,  distinct  types  of  P-O  fit.  First,  it  is 
conceptualized  as  a  fit  between  the  employees’  needs  and  values  and  the  opportunities 
provided  by  the  organization  in  which  they  operate.  A  second  conceptualization  is  as  a 
fit  between  the  demands  of  the  organization  and  the  abilities  of  the  person  to  meet  those 
demands  (Kulik  et  al.,  1987).  One  approach  to,  or  conceptual  model  of,  P-O  fit  is  job 
characteristics  theory,  which  emphasizes  the  importance  of  a  fit  between  characteristics 
of  the  job  and  characteristics  of  the  jobholder  (Hackman  &  Oldham,  1976).  This 
approach  has  also  been  referred  to,  and  studied  as  person-job  (P-J)  fit  or  the 
compatibility  of  individuals  with  the  specific  tasks  they  are  expected  to  perform  (Kristof, 
1996),  or  as  the  desires  of  a  person  and  the  attributes  of  a  job  (Edwards,  1991). 

Research  has  shown  that  P-J  fit  is  related  to  multiple  organizationally  relevant 
outcomes  (Kristof,  1996),  including  work  attitudes  (e.g.,  Dawis  &  Lofquist,  1984),  work 
performance  (e.g.,  Murphy,  1993),  and  intention  to  quit  or  turnover  (e.g.,  Bretz  &  Judge, 
1994;  Schneider,  1987).  Although  researchers  have  examined  the  effects  of  fit  on  several 
outcomes,  most  studies  focus  on  positive  outcomes  for  the  individuals.  For  example, 
Bretz  and  Judge  (1994)  found  powerful  direct  effects  of  P-O  fit  on  organizational 
satisfaction.  Specifically,  they  found  these  effects  when  P-O  fit  was  conceptualized  in 
various  ways,  such  as  the  extent  to  which  an  individual’s  characteristics  (i.e.,  knowledge, 
skills,  abilities  and  other  employee  characteristics,  KSAOs)  met  job  requirements  (i.e., 
indicative  of  P-J  fit),  and  the  degree  to  which  individuals’  values  aligned  with  the 
organization’s  (i.e.,  value  congruence).  Various  conceptualizations  of  P-J  fit  have  also 
been  found  to  be  predictive  of  intentions  to  quit  (Kristof,  1996).  A  vast  compendium  of 
research  has  suggested  that  individuals  whose  interests  are  incongruent  with  their  jobs 
are  more  apt  to  leave  their  employing  organizations  than  their  congruent  counterparts 
(Heilman,  1998;  Jackofsky  &  Peters,  1983;  Spencer  &  Steers,  1981;  Spokane,  1985). 
Finally,  self-report  and  more  objective  measures  have  been  used  to  assess  the  effects  of 
P-O  fit  on  individuals’  work  performance.  For  example,  Bretz  and  Judge  (1994) 
investigated  fit  as  a  predictor  of  career  success.  Their  results  indicated  indirect  effects  of 
P-O  fit  on  job  promotions  and  to  a  lesser  extent  salary  level,  in  addition  to  direct  effects 
on  organizational  tenure  and  satisfaction. 
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Researchers  in  various  disciplines  have  addressed  the  need  to  develop  techniques  to 
accomplish  sufficient  P-0  matching.  However,  the  unique  requirements  and  various 
practical  and  research  problems  encountered  within  each  discipline  shape  the 
procedures  developed  to  meet  those  problems.  For  example,  within  military 
classification,  methodologies  have  been  adapted  to  address  special  problems  raised  by 
enlistment  procedures  and  by  the  sequential  nature  of  military  accessioning  (Kroeker, 
1989).  Job  options  ranging  from  optimal  to  least  acceptable  must  be  identified  rapidly 
for  each  potential  recruit. 

Although  many  studies  indicate  that  fit  influences  individuals’  preferences  and 
organizations’  selection  and  classification  decisions,  there  is  not  a  clear  understanding 
of  how  specific  procedures  affect  levels  of  P-0  fit  (Karren  &  Graves,  1994).  For  instance, 
strategies  that  strive  to  present  realistic  organizational  previews  may  promote  higher 
levels  of  fit  than  more  general  recruitment  practices  (Kristof,  1996).  Moreover,  it  has 
been  suggested  that  the  greater  the  technical  job  requirements  the  greater  the 
importance  of  P-J  fit  (Borman  et  al.,  2002;  Werbel  &  Gilliland,  1999).  Because  it  may  be 
difficult  for  Navy  applicants  to  assess  the  available  jobs  in  the  organization,  a  particular 
effort  must  be  made  to  increase  their  salience  to  applicants.  Another  relatively 
unexplored  area  regarding  P-0  fit  is  the  improvement  of  fit  assessments  made  during 
the  selection  and  classification  process  (Werbel  &  Gilliland,  1999).  Recruiters  and  career 
counselors  who  are  able  to  clearly  convey  aspects  of  the  organization  may  aid  recruits  in 
more  accurately  determining  levels  of  P-J  fit.  Moreover,  implementing  interest 
assessment,  personality  tests,  and  other  means  of  measuring  non-cognitive 
characteristics  may  help  to  recognize  applicant  qualities  that  reflect  a  “good  fit”  with  the 
organization  (Kristof,  1996;  Tziner,  Meir,  &  Segal,  2002).  Similar  to  realistic  job 
previews  (e.g.,  Rynes,  1991),  using  these  methods  to  improve  early  fit  assessments  can 
save  individual  and  organizational  resources  by  improving  screening  mechanisms  and 
individuals’  self-selection  out  of  particular  occupations  (Werbel  &  Gilliland,  1999). 

The  Navy  has  established  programs  of  research  to  investigate  and  improve  P-J  fit 
through  selection  and  assignment  procedures  (Borman  et  al.,  2002).  These  programs 
include  the  recurrent  investigation  of  the  validity  of  the  Armed  Service  Vocational 
Aptitude  Battery  (ASVAB)  and  its  use  (Foley  &  Rucker,  1989),  and  the  development  of 
new  classification  algorithms  that  optimize  matching  (Watson,  2002).  Central  to  the 
topic  of  this  research  project,  recent  Navy  initiatives  have  proposed  the  examination  of 
the  usefulness  of  individual  interest  assessment  and  other  alternative  employment 
predictors  (Borman  et  al.,  2002;  Lightfoot,  Alley,  Schultz,  &  Watson,  2000;  Lightfoot  et 
al.,  1999;  NPRST,  1998). 

For  decades,  the  vocational  counseling  literature  has  made  the  convincing  empirical 
argument  that  job  selection  based  on  stated  preferences  or  “interests”  may  have  a 
beneficial  impact  for  the  individual,  in  terms  of  job  satisfaction  (Betz,  2001;  Borman  et 
al.,  2002;  Dawis,  1991).  Career  decisions  are  among  the  most  important  decisions  an 
individual  must  make  (Gati,  1998).  The  career  decision-making  process  consists  of  the 
prescreening  of  viable  choices,  an  in-depth  exploration  of  the  most  promising 
alternatives,  and  ranking  these  alternatives.  The  ultimate  goal  of  this  process  is  to 
maximize  P-0  fit  (Gati  &  Asher,  2001;  Holland,  1985;  Lofquist  &  Dawis,  1984, 1991). 
Again,  the  P-0  perspective  assumes  that  congruence  between  an  individual’s  interests 
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and  the  characteristics  of  his  or  her  chosen  occupation  results  in  positive  outcomes  for 
the  individual  as  well  as  the  organization  (Dawis  &  Lofquist,  1984;  Holland,  1985; 
Spokane,  1985).  Advances  in  the  last  decade  have  shown  that  we  can  reliably  measure 
the  interest  facet  of  human  behavior  and  that  under  certain  conditions  these  can  add 
substantially  to  our  ability  to  predict  relevant  outcomes  (Borman  et  al.,  2002;  Savickas, 
1999). 

Vocational  I  nterest  Measurement 

For  almost  a  century,  industry  has  recognized  the  financial  worth  of  having  a  person 
interested  in  his  or  her  job.  Psychologists  in  the  1920s  began  to  explore  the  possibility  of 
interest  measurement  as  an  attempt  to  supplement  existing  measures  of  special  and 
general  abilities  (Hansen,  1994;  Hunter  &  Hunter,  1984;  Roe  &  Seligman,  1964).  There 
have  been  numerous  studies  investigating  the  nature  of  vocational  interests  and,  in 
particular,  developing  inventories  to  measure  those  interests  (e.g.,  Strong,  1955;  Super, 
1949). 

Common  paper-and-pencil  interest  measures  include  the  Strong  Vocational  Interest 
Blank  and  its  various  revisions  (SVIB,  Hansen  &  Campbell,  1985;  Harmon,  Hansen, 
Borgen,  &  Hammer,  1994),  the  Kuder  Occupational  Interest  Survey  (KOIS;  Kuder,  1964; 
Kuder  &  Zytowski,  1991),  and  the  Self-Directed  Search  (SDS;  Holland,  Powell,  & 
Fristsche,  1994).  Vocational  interest  inventories  such  as  these  require  an  applicant  to 
indicate  his  or  her  preference  (e.g.,  like,  indifferent,  or  dislike)  for  a  variety  of  activities 
or  job  titles.  For  some  time,  such  inventories  have  been  the  tool  that  industry  has 
effectively  utilized  for  job  placement  of  prospective  employees  (Hansen,  1994). 

Over  the  last  70  years  of  interest  measurement,  there  have  been  3  prominent 
strategies  employed  in  the  development  of  interest  scales  (Burisch,  1984).  One  of  these 
methods,  the  external  approach,  is  based  on  the  premise  that  people  can  be  sorted  into 
groups.  This  method  of  scale  development  involves  contrasting  item  responses  of  an 
occupational  sample  with  those  of  a  general  group.  Early  interest  scales  were  empirically 
derived,  heterogeneous  scales  composed  of  items  that  separated  workers  in  specific 
occupational  groups  (Harmon,  1999).  Scales  developed  using  this  approach  are  difficult 
to  fake,  and  are  very  efficient  for  predicting  membership  in  specific  occupations. 
However,  this  approach  typically  produces  longer  scales,  that  take  longer  to  administer, 
and  subsequently  are  more  costly.  The  internal  approach  to  scale  development  involves 
developing  homogenous  scales  from  a  large  number  of  interest  items  using  some  type  of 
clustering  technique  (Harmon,  1999).  A  structural  analysis  of  item  response  data,  such 
as  factor  analysis,  is  performed  to  identify  the  underlying  independent  dimensions  of 
vocational  interest.  A  major  advantage  of  this  strategy  is  that  the  meaning  of  scale  scores 
is  easy  to  communicate  to  clients,  because  typically,  the  label  of  the  scale  indicates  the 
content  and  definition  of  the  scale  (Savickas,  1999).  Moreover,  the  derived  scales  are 
relatively  independent,  thus  they  can  be  used  to  predict  occupational  membership.  One 
final  approach,  the  rational  approach  to  interest  measurement,  is  based  on  the  idea  that 
a  theory  or  model  of  vocational  interests  should  influence  the  choice  and  definition  of 
constructs  (or  scales),  and  precede  the  development  of  the  items  (Betz,  2001).  Studies 
examining  these  three  strategies  for  vocational  interest  scale  construction  have  shown 
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that  they  produce  equally  valid  scales  (Betz,  2001;  Burisch,  1984).  Preference  for 
employing  one  or  more  of  these  approaches  is  typically  based  on  other  criteria  such  as 
assessment  purpose,  economics,  etc.  (Lightfoot  et  al.,  1999). 

These  and  other  strategies  have  been  employed  by  scientists  and  have  resulted  in 
dozens,  if  not  hundreds,  of  vocational  interest  inventories.  These  instruments  vary 
widely  in  sophistication,  extent  of  use,  and  psychometric  quality.  For  today’s  Navy  it 
seems  that  the  most  appropriate  approach  to  interest  measurement,  based  on  past 
research  and  current  objectives,  would  be  a  blend  of  the  external  and  internal  strategies 
(Lightfoot  et  al.,  1999).  Prior  to  this  research  project,  several  steps  had  been  taken  to 
incorporate  interest  measurement  in  the  Navy’s  classification  system. 

Previous  researchers  evaluated  existing  interest  inventories  for  Navy  use  (Lightfoot 
et  al.,  1999),  and  consequently  the  lack  of  suitable  alternatives  prompted  other  Navy 
researchers  to  develop  a  rather  exhaustive  vocational  interest  inventory  (Lightfoot  et  al., 
2000).  This  inventory  was  deemed  objectionable  for  several  reasons,  the  most  salient  of 
which  was  the  overwhelming  number  of  items  (i.e.,  more  than  500  activity  statements). 
During  Navy  classification  each  recruit  must  undergo  a  battery  of  aptitude  and  medical 
tests,  as  well  as  other  administrative  procedures,  which,  in  aggregate,  make  the 
administration  of  an  exhaustive  inventory,  within  such  a  limited  period  of  time, 
problematic.  Thus,  in  designing  the  inventory  for  the  current  study,  one  focus  was  the 
number  of  items  the  instrument  contained.  It  was  determined  that  the  optimal 
vocational  interest  inventory  for  Navy  recruits  would  include  a  minimal  number  of 
items  while  maintaining  an  appreciable  degree  of  predictive  utility. 

In  addition,  the  optimal  inventory  would  be  required  to  meet  several  guidelines  to 
better  incorporate  the  instrument  in  the  classification  system.  For  example,  the 
inventory  should  adequately  discriminate  among  ratings  for  the  purpose  of  person-job 
matching  and  add  to  the  predictive  effectiveness  of  ability  measures  alone.  Researchers 
have  suggested  that  the  Navy  interest  inventory  should  be  designed  to  measure 
vocational  interests  that  reflect  the  specific  work  of  Navy  ratings  and  the  unique  aspects 
of  the  Navy’s  work  environment  (Lightfoot  et  al.,  1999).  One  additional  focus  of  this 
project  was  to  adequately  inform  applicants  about  the  specific  nature  of  Navy  work 
using  text  and  pictures.  In  describing  the  Navy  world-of-work,  written  tests,  as  those 
mentioned  above  require  a  certain  amount  of  reading  comprehension  and  verbal  ability 
on  the  part  of  the  individual  completing  the  inventory.  This  may  be  a  cause  for  concern 
as  to  the  value  of  these  paper-and-pencil  tests  in  the  assessment  of  interests  of  those 
individuals  who  possess  comparatively  low  mental  ability  or  insufficient  reading  skills. 
As  one  solution  to  this  problem,  the  current  research  paper  centers  on  the  development 
of  an  interest  inventory  that  utilizes  a  large  number  of  digital  images  to  comprise 
individual  interest  items.  There  is  some  evidence  that  pictorial  presentation  has  an 
advantage  over  the  usual  verbal  measures  when  used  with  certain  individuals  who  may 
tend  to  misconstrue  the  meaning  of  printed  statements;  such  as  those  with  limited 
schooling,  inferior  reading  ability,  or  cultural  disadvantage  (Jastak  &  Jastak,  1970).  This 
mode  of  interest  assessment  was  also  chosen  so  as  to  provide  those  applicants  with  little 
prior  knowledge  of  Navy  jobs  an  extensive  amount  of  information  beyond  that  of  written 
items  alone.  Finally,  the  inventory  needs  to  be  based  on  a  flexible  model  that  may  be 
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readily  altered  as  the  organization  and  specific  jobs  change.  Currently,  there  are  no 
inventories,  including  those  previously  developed  for  the  Navy,  that  meet  all  of  the 
aforementioned  design  criteria. 

Interest  measurement  has  been  employed  in  selection  and  classification  with  the 
goal  of  maximizing  predictions  of  success,  satisfaction,  and  persistence  in  training  as 
well  as  on  the  job  (Hansen,  1994).  There  is  a  strong  belief  in  the  literature  that  interests 
or  preferences  should  predict  the  choice,  selection,  and  attainment  of  success  in 
particular  activities  by  an  individual  (Barak,  1981).  Largely,  the  use  of  interest  measures 
in  the  context  of  selection  and  classification  is  based  on  the  assumption  that  knowing  a 
person’s  motivation  or  the  manner  in  which  an  individual  responds  to  a  variety  of 
situations  can  help  predict  success  in  a  job  (Vroom,  1964).  The  idea  that  vocational 
satisfaction  is  related  to  the  degree  of  fit  or  congruence  between  the  attributes  of  an 
individual  and  the  work  environment  is  central  to  the  field  of  vocational  counseling 
(Tranberg  et  al.,  1993).  It  is  generally  recognized  that  work-relevant  abilities  (e.g., 
technical  aptitude),  interests,  and  values  are  primary  considerations  when  helping 
persons  with  career  exploration  (Barak,  2001;  Betz,  2001).  Work-relevant  abilities 
include  non-cognitive  abilities  in  addition  to  the  customary  cognitive  abilities 
(Mumford,  Peterson,  &  Childs,  1999).  Interest  assessment  would  make  an  important 
contribution  to  any  placement  or  selection  procedure  when  one  considers  the  large 
number  of  young  people  who  are  undecided  in  their  career  goals  and  who  typically  have 
little  knowledge  regarding  specific  occupations  (Hansen,  1994).  Finally,  the  assessment 
of  interests  would  reduce  the  necessity  of  the  individuals  knowing  specific  tasks  involved 
in  various  occupations  (Wilbourn  &  Alley,  1978). 


Selection  and  Classification  Programs 

The  purpose  of  a  selection  program  is  to  identify  the  best  individuals  to  perform  a 
job  within  the  organization.  Gathering  information  about  the  job  in  the  organization,  or 
job  analysis,  is  the  logical  starting  point  in  the  development  of  such  a  program 
(Gatewood  &  Feild,  1998).  This  information  typically  describes  the  environment,  the 
work  activities,  and  the  outcomes  that  characterize  the  job.  This  information  is  essential 
to  selection  and  classification  programs  for  several  reasons.  First,  this  information 
conveys  to  potential  applicants  information  about  the  nature  and  demands  of  the  job 
(Schippmann,  1999).  Second,  this  information  is  also  important  because  it  indicates 
what  constitutes  successful  job  performance  (Schippmann,  1999).  Finally,  this 
information  allows  organizations  to  identify  the  knowledge,  skills,  abilities,  and  other 
employee  characteristics  (KSAOs)  that  a  worker  should  possess  in  order  to  perform  the 
job  successfully  (Gatewood  &  Feild,  1998).  In  general,  where  selection  is  concerned, 
organizations  are  by  and  large  interested  in  abilities,  which  become  the  basic 
characteristics  to  be  evaluated  in  applicants  (Gatewood  &  Feild,  1998). 

The  assumption  in  selection  is  that  applicants  possess  different  amounts  of  job- 
related  abilities  (i.e.,  KSAOs)  necessary  for  successful  performance.  The  purpose  of 
assessment  is  to  measure  these  differences,  and  it  is  in  this  way  that  promising 
applicants  can  be  distinguished  from  unqualified  applicants.  On  the  other  hand,  the 
crucial  challenge  in  classification  is  maintaining  assignment  procedures  so  that  a  close 
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match  between  human  skills  and  job  requirements  is  achieved  (Kroeker,  1989).  When 
classification  is  done  poorly,  the  consequences  are  more  costly  and  complex.  For 
example,  with  regard  to  the  Navy  training  school,  attrition  may  be  elevated  and  school 
seats  underutilized.  Moreover,  after  training  poor  incumbent  performance  may  affect 
readiness  and  job  satisfaction,  substantially  reducing  retention  rates.  During  job 
assignment  it  may  be  useful  to  facilitate  the  matching  process  by  examining  other 
employee  characteristics,  such  as  an  applicant’s  job  interests  or  work  preferences,  in 
addition  to  cognitive  abilities  (Borman  et  al.,  2002;  Hunter  &  Hunter,  1984;  NPRST, 
1998). 

Ordinarily  the  first  activity  in  the  Navy’s  selection  process  is  the  recruiter  interview, 
during  which  preliminary  testing  is  conducted  with  each  applicant  to  identity  and 
remove  those  not  suited  for  the  service.  This  screening  process  includes  a  determination 
of  applicant  citizenship  status,  education  level,  moral  character,  general  physical 
condition,  and  the  administration  of  the  Enlistment  Screening  Test  (EST)  to  pre-screen 
prospects  on  verbal  and  mathematical  ability  (Foley  &  Rucker,  1989).  Next,  qualified 
applicants  move  on  to  the  Military  Entrance  Processing  Station  (MEPS),  or  a  test  site 
where  full-scale  enlistment  processing  is  accomplished  (e.g.,  medical  examination, 
counseling,  and  work  assignment).  At  the  MEPS  applicants  are  given  a  more  stringent 
screening  measure,  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB).  The 
ASVAB  consists  of  a  battery  of  tests  covering  four  content  domains:  verbal,  numerical, 
technical,  and  perceptual  speed  (NPRST,  1998).  By  adding  scores  across  tests  the 
applicant  is  qualified  for  service,  assigned  to  a  technical  school,  and  enlisted  into  a  Navy 
career;  all  in  roughly  three  hours  (NPRST,  1998;  Schratz  &  Ree,  1989). 

Subsequent  to  those  screening  activities  pertinent  to  selection,  all  the  services 
address  the  issue  of  classification  into  an  occupational  area  or  a  specific  job.  Typically, 
the  actual  assignment  of  recruits  to  training  schools,  and  thus  jobs,  is  accomplished  via 
computerized  person-job  match  algorithms.  Each  service  has  its  own  algorithm,  which 
reflects  its  current  policies  toward  the  relative  priorities  of  filling  jobs  at  any  point  in 
time  (Kroeker,  1989). 

Classification  efforts  are  optimized  when  jobs  are  examined  for  specific 
requirements,  and  certain  individual  characteristics  are  assessed  to  ascertain  if  the 
person-job  match  would  be  maximized  (NPRST,  1998).  All  of  the  classification 
algorithms  used  by  the  individual  services  adhere  to  a  minimum  aptitude  standard  for 
each  job,  and  all  are  designed  to  maximize  the  utilization  of  training  school  vacancies 
across  jobs  (Kroeker,  1989).  The  benefits  of  including  intellectual  measures  (e.g.,  the 
ASVAB)  for  both  selection  and  classification  decisions  are  undeniable  (Hunter  & 
Hunter,  1984).  The  benefits  for  the  organization  include  reduced  training  failures  and 
training  costs,  and  greater  performance  and  productivity.  For  the  individual,  the 
benefits  include  a  greater  probability  of  success  in  training  and  on-the-job  performance. 

A  second  characteristic  that  may  be  evaluated  to  maximize  P-J  fit  is  an  individual’s 
vocational  interest.  Again,  both  the  organization  and  the  individual  (e.g.,  increased  job 
satisfaction)  would  stand  to  benefit  from  classification  decisions  based  in  some  part  on 
interests  (Borman  et  al.,  2002).  Since  the  move  to  an  all-volunteer  force,  occupational 
preference  has  assumed  a  larger  role  in  military  classification  (Schratz  &  Ree,  1989). 
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Individuals’  preferences  have  to  be  considered,  to  some  degree,  in  job  assignments. 
Relative  to  other  branches  of  service  (i.e.,  Air  Force  and  Army),  the  Navy  makes  a  weak 
attempt  to  include  preference  scores  in  their  assignment  system. 

Currently,  this  minimal  effort  to  assess  preference  consists  of  providing  the  applicant 
with  a  list  of  jobs  or  broader  occupational  groups  (e.g.,  administration,  health  care, 
ordnance,  etc.)  and,  in  some  cases,  with  laminated  job  cards  displaying  limited  amounts 
of  information  regarding  possible  occupations  (Russell,  Knapp,  &  Campbell,  1992). 

Some  of  the  individuals  are  afforded  the  time  available  during  processing  to  review  this 
information,  and  then  they  are  asked  to  indicate  their  top  five  occupational  group 
preferences  (Russell  et  al.,  1992).  In  other  cases,  the  classifier  may  indicate  irrelevant 
preferences  so  that  the  remainder  of  the  computerized  accessioning  procedure  can  be 
completed.  A  reasonable  assumption  is  that  the  current  methods  used  to  determine 
preference  lack  structure,  are  insufficient,  and  may  not  lead  to  reliable  or  accurate 
preference  data.  In  many  cases  job  selection  is  made  with  little  knowledge  of  actual  job 
content,  which  may  lead  to  dissatisfaction  in  the  selected  career.  Even  when  job 
descriptions  are  available,  the  technical  wording  of  the  descriptions  may  render  them 
useless  for  enlistees  with  limited  reading  abilities  (Bartol,  1981). 

The  Navy  is  continuously  faced  with  constraints  that  limit  the  usefulness  of  optimal 
classification  strategies.  In  all  organizations  the  work  assignment  methods  must  operate 
under  one  or  more  constraints,  such  as  budget  limitations,  training  seat  availability, 
group  goals,  management  priorities,  and  applicant  preferences  (Kroeker,  1989). 
Generally,  the  existence  of  constraints  reduces  potential  gains  from  classification.  In  an 
attempt  to  address  the  Navy’s  deficiencies  in  its  classification  process,  Navy  researchers 
have  worked  with  other  contributors  to  develop  and  test  new  classification  decision 
support  software,  the  Rating  Identification  Engine  (RIDE),  which  provides  classification 
algorithms  for  enlisted  personnel.  The  goal  behind  the  new  system  is  to  improve  the 
recruit-rating  assignment  process  so  that  it  provides  greater  utility  in  the  operational 
classification  system  (Watson,  2002).  The  assignment  system  utilizes  an  algorithm 
comprised  of  several  components,  such  as  cognitive  ability,  predicted  training  success, 
job  priority,  and  probability  of  attrition.  The  system  then  generates  a  number  reflecting 
the  value  of  assigning  the  recruit  to  each  rating  (e.g.,  such  as  Machinist’s  Mate  [MM]). 
Navy  ratings  with  no  openings,  or  for  which  the  applicant  is  not  qualified  are  eliminated, 
and  the  remaining  jobs  are  ranked  accordingly. 

To  meet  future  manpower  requirements,  the  Navy  envisions  using  RIDE  to  first 
identify  appropriate  jobs  for  an  individual  applicant  (NPRST,  1998).  Then,  an  interest 
inventory  (i.e.,  JOIN)  would  be  utilized  to  filter  those  jobs  in  which  a  person  is  likely  to 
be  interested.  It  is  purported  that  the  simultaneous  use  of  an  interest  measure  and  the 
RIDE  system  will  improve  the  match  between  the  recruit’s  abilities  and  interests,  and 
his  or  her  assigned  rating.  This  procedure  would  enable  applicants  and  classifiers  to 
discuss  job  options  and  make  a  final  decision  about  enlistment  in  the  brief  time  frame 
allotted.  Ultimately,  this  system  should  serve  as  a  means  of  increasing  job  satisfaction, 
performance,  and  retention  (Watson,  2002). 
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Abilities  and  I  nterests 

The  field  of  psychology  has  provided  many  of  the  tools  (e.g.,  tests)  that  serve  as  a 
critical  element  in  the  selection  and  classification  process  (Schratz  &  Ree,  1989). 
Aptitude  or  ability  tests  are  used  to  measure  how  well  an  individual  can  perform  specific 
parts  of  a  job  (Gatewood  &  Feild,  1998).  Additionally,  ability  tests  are  differentiated  by 
the  nature  of  the  content  they  measure  (e.g.,  specific  vs.  general)  and  the  breadth  of 
topics  covered  (e.g.,  intellectual,  mechanical,  spatial,  perceptual,  motor,  etc.). 

Many  of  the  research  paradigms  in  ability  testing  focus  on  improving  the  prediction 
of  the  outcomes  of  both  education  (or  training)  and  job  performance  (Dillon,  1989). 
Research  in  military,  as  well  as  civilian  organizations  has  shown  that  a  lack  of 
qualifications  can  lead  to  training  failures  and  poor  job  performance.  For  example,  a 
high  school  diploma  has  been  shown  to  be  the  best  single  predictor  of  adaptability  to 
military  life  (Foley  &  Rucker,  1989).  The  first  organized  testing  program  of  mental 
ability  was  developed  in  response  to  the  need  to  assess  a  large  number  of  men  for  the 
military  mobilization  of  World  War  I  (WWI).  From  that  point  forward  the  military  has 
been  at  the  forefront  of  psychological  testing  (Schratz  &  Ree,  1989).  One  of  the  earliest 
mental  ability,  or  intelligence  tests,  was  developed  in  1917  to  provide  scores  that  could 
be  used  to  reject  recruits  who  were  thought  to  be  unfit  for  military  service  during  WWI 
(Gatewood  &  Feild,  1998).  It  was  called  the  Army  Alpha  and  over  one  million  men  were 
tested  across  the  United  States.  The  use  of  this  test  generated  interest  in  the 
development  of  other  ability  tests  for  use  in  vocational  counseling  and  industrial 
selection.  Furthermore,  during  World  War  II  all  three  military  organizations  had 
extensive  psychological  testing  programs  that  emphasized  developing  specialized  tests 
to  assist  in  placing  recruits  in  the  most  appropriate  jobs  (Gatewood  &  Field,  1998). 

The  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  was  introduced  by  the 
Department  of  Defense  (DOD)  in  1968  and  later  adopted  by  the  Navy,  and  all  other 
military  branches,  for  selecting  enlistees  and  classifying  them  into  military  occupations 
(Foley  &  Rucker,  1989).  Up  to  January  2002,  the  ASVAB  was  comprised  of  a  number  of 
subtests  that  include  arithmetic  reasoning,  auto  and  shop  information,  coding  speed, 
electronics  information,  general  science,  mathematics  knowledge,  mechanical 
comprehension,  numerical  operations,  paragraph  comprehension,  and  word  knowledge. 
After  January  2002,  the  ASVAB  was  updated  to  remove  the  numerical  operations  and 
coding  speed  subtests  and  add  an  assembling  objects  subtest.  The  ASVAB  is  regularly 
updated,  approximately  every  four  years,  in  order  to  increase  the  security  of  the  test 
items,  to  allow  for  the  replacement  of  obsolete  test  items,  and  to  incorporate 
improvements  in  the  field  of  measurement  into  the  new  tests.  Classification  of  Navy 
recruits  is  accomplished  by  using  11  composites  of  ASVAB  subtests  that  predict  success 
in  the  associated  initial  occupational  training  school  (Foley  &  Rucker,  1989). 

Assessment  of  the  ASVAB  has  been  conducted  by  a  number  of  researchers  over  the  test’s 
history  (Foley  &  Rucker,  1989). 

Although  the  ASVAB  has  oft  been  referenced  as  an  exemplary  psychometric 
instrument  throughout  the  years,  it  has  not  been  without  criticism.  Several  researchers 
have  suggested  that  the  ASVAB  has  several  less  than  desirable  characteristics.  For 
example,  Jensen  (1985)  criticized  the  lack  of  differential  validity  for  the  ASVAB  subtests. 
Essentially,  the  composites  formed  are  highly  inter-correlated,  suggesting  that  each 
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subtest  does  not  account  for  some  unique  proportion  of  the  variance.  Moreover,  Jensen 
suggested  that  the  instrument  is  of  limited  utility  with  regard  to  assisting  individuals  in 
choosing  between  competing  job  alternatives.  Similarly,  Murphy  (1984)  commented  on 
the  high  inter-correlations  between  composite  scores,  and  the  inadequacy  of  the 
instrument  in  providing  the  type  of  information  that  would  allow  classifiers  to 
differentiate  among  vocational  aptitudes  and  place  recruits  in  the  most  suitable 
schools/service  careers. 


Purpose  and  Objectives 

The  purpose  of  the  current  study  was  to  develop  and  test  pictorial  items  for  an 
interest  instrument  that  could  easily  be  implemented  into  the  Navy’s  current 
classification  process,  and  be  used  with  the  RIDE  system.  The  challenge  was  to  develop 
a  flexible  computer-administered  inventory  that,  by  design,  would  assess  applicant 
preferences  and  interest.  Specific  considerations  were  the  inventory’s  length  (i.e.,  time 
taken  to  administer),  the  ability  to  easily  alter  its  content  as  Navy  jobs  changed,  and  its 
ability  to  be  used  by  naive  applicants.  Additionally,  the  inventory  would  not  only 
measure  an  applicant’s  interest  in  different  dimensions  of  Navy  enlisted  jobs  but  also 
inform  the  applicant  regarding  the  Navy’s  world-of-work. 

The  first  objective  was  to  collect  job-related  information  concerning  each  enlisted  job 
in  order  to  develop  a  model  (or  structure)  to  serve  as  the  basis  for  individual  interest 
items.  One  goal  was  to  identify  the  work  activities,  work  styles,  and  work  environments 
that  are  most  generic,  and  as  such,  would  likely  be  applicable  to  multiple  ratings  while 
maintaining  an  accurate  description  of  each  of  the  analyzed  ratings.  Another  objective, 
relative  to  the  first,  was  to  develop  pictorial  interest  items  that  represent  each  of  the 
previously  mentioned  dimensions  of  Navy  jobs.  Job  analysis  data  collected  from  subject 
matter  experts  (SMEs)  were  analyzed  to  examine  the  structure  of  the  model  driving  the 
instrument  and  the  content  of  the  inventory  items.  The  aim  was  to  develop  an  interest 
structure  that  was  unique  in  that  the  items  distinguished  among  different  ratings,  but 
also  general  enough  so  that  traditional  clusters  of  Navy  jobs  resembling  existing 
occupational  groups  could  be  formed. 

Following  instrument  development,  the  next  objective  was  to  gather  responses  from 
a  group  of  new  recruits,  during  basic  training,  to  evaluate  the  psychometric  properties  of 
the  pictorial  items  and  content  of  the  computer-administered  instrument.  Assessment 
of  the  instrument  was  conducted  on  the  item  level  as  well  as  at  the  scale  level.  The  aim 
was  to  develop  individual  items  that  would  elicit  adequate  variance  in  participants’ 
responses  and  adequate  internal  consistency  reliability,  across  and  within  individual 
items.  Exploratory  factor  analyses  of  participants’  responses  were  conducted  to 
determine  the  potential  grouping  of  items  into  interpretable  scales.  Additional  analyses 
were  also  performed  to  investigate  if  any  differences  existed  in  participants’  interests 
when  gender  and  race  were  used  as  grouping  variables. 

One  final  objective  was  the  evaluation  of  the  usability  of  the  computer-administered 
instrument  using  recruits  to  determine  the  programs  functionality  under  certain 
organizational  constraints  (e.g.,  available  test  time,  recruit  characteristics).  In  part,  the 
information  collected  was  used  to  verify  that  recruits  possess  the  skills  necessary  to 
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complete  the  alternative  mode  of  administration  in  an  acceptable  amount  of  time. 
Furthermore,  this  information  provided  the  opportunity  to  collect  general  feedback, 
ratings  regarding  the  instruments  usability  (e.g.,  ease  of  use,  clarity  of  instructions,  etc.), 
and  opinions  regarding  the  general  appeal  of  the  device  (e.g.,  visual  appeal,  comparison 
to  a  paper-and-pencil  format,  etc.). 

Aside  from  the  obvious  advantage  to  the  Navy  of  such  an  instrument,  the  current 
study  contributes  to  the  existing  vocational  interest  literature  in  several  ways.  First,  this 
study  provides  an  examination  of  a  computer-administered  inventory  that  centers  on 
the  use  of  pictures  as  individual  interest  items.  Several  authors  have  expressed  the  need 
to  further  explore  and  utilize  non-traditional  administration  methods,  as  well  as  non- 
conventional  items  to  assess  interests  (Lent,  2001;  Russell,  2001;  Savickas,  2001). 
Furthermore,  this  study  corresponds  to  the  vision  that  has  been  expressed  in  the  field  of 
vocational  psychology  to  renew  linkages  with  industrial-organizational  psychology 
(Gottfredson,  2001).  Specifically,  this  research  effort  merges  the  study  of  vocational 
interests  and  developments  in  job  analysis,  personnel  selection,  and  classification. 


Method 


Phase  I :  I  nstrument  Development 

The  premise  leading  to  the  development  of  an  interest  model  and  inventory  items 
was  based  in  part  on  the  recommendations  offered  in  several  preliminary  reports 
(Lightfoot  et  al.,  2000;  Lightfoot  et  al.,  1999;  Watson,  Hindelang,  &  Michael,  2002).  In 
particular,  Watson  and  his  colleagues  (2002)  suggested  the  design  criteria  that  guided 
the  current  research  project  and  first  identified  the  inventory  as  Jobs  and  Occupational 
Interest  in  the  Navy  (JOIN). 

Model  Development 

The  development  of  the  JOIN  interest  model  began  with  the  examination  of  Navy 
jobs  guided  by  a  basic  hierarchical  model  of  work  developed  by  Schippmann  (1999). 
According  to  the  proposed  framework,  at  the  macro-level,  the  organization  (i.e.,  the 
Navy)  consists  of  various  job  families  or  a  grouping  of  jobs  that  are  categorized 
according  to  organizational  function  or  work  process  (e.g.,  aviation  area,  health  care 
area,  etc.).  The  world  of  Navy  work  can  also  be  examined  at  a  micro  level,  in  terms  of 
those  work  activities  or  tasks  describing  work  performed,  which  are  subsumed  under  a 
superordinate  job  family.  A  variation  on  basic  job  analytic  methods  was  utilized  to 
collect  these  organizational  data. 

The  first  step  involved  the  collection  of  all  of  the  available  enlisted  job  descriptions 
from  several  Navy  resources;  such  as  the  U.S.  Navy  homepage,  Navy  Personnel 
Command  homepage,  and  job  information  cards.  Each  job  description  was  reviewed  and 
words  that  reflected  basic  work  dimensions  were  highlighted.  Specific  components 
included  community  areas,  work  styles,  work  environments,  and  work  activity 
statements.  From  these  highlighted  extracts,  preliminary  items  were  created  that 
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seemed  most  representative  of  each  of  the  79  Navy  enlisted  ratings.  With  regards  to 
work  activities,  these  statements  were  further  reduced  to  two  components,  a  process 
(e.g.,  verbs  such  as  analyze,  maintain,  operate)  and  a  specific  content  (e.g.,  nouns  such 
as  documents,  mechanical  equipment,  weapons,  etc.).  The  purpose  in  defining  job  tasks 
as  process-content  (PC)  pairs  was  threefold.  First,  it  would  allow  very  specific  activities 
to  be  more  easily  conveyed  to  naive  recruits  who  may  not  have  an  understanding  of  the 
detailed  content.  For  example,  many  job  tasks  involve  the  use  of  equipment  or 
machinery  that  is  not  common  knowledge;  such  as  pneumatic  devices,  launch  recovery 
craft,  etc.  By  using  broader  terms  to  describe  such  devices  a  more  precise  level  of 
interest  in  the  activities  could  be  assessed.  Second,  describing  work  activities  in  this 
manner  would  allow  for  a  reduction  in  the  total  number  of  items  needed.  For  example,  a 
large  number  of  activities  may  include  similar  types  of  equipment  (e.g.,  hydraulics, 
pneumatics,  steam  powered,  etc.),  and  thus  could  be  subsumed  under  one  category  (e.g., 
mechanical  equipment).  Finally,  as  Navy  jobs  change  and  the  tasks  within  those  jobs 
change,  the  inventory  items  could  be  revised  without  much  difficulty.  For  example, 
subsequent  large-scale  validation  of  the  new  items  would  not  have  to  be  conducted. 
Rather  subject  matter  experts  (SMEs)  could  be  utilized  to  judge  new  tasks  and  place 
them  in  the  appropriate  PC  pair  categories.  These  job  activities  would  serve  as  the  core 
items  in  the  inventory.  For  example,  some  of  the  activities  created  included:  repair 
mechanical  equipment,  analyze  documents,  direct  aircraft,  make  facilities,  etc.  The 
primary  focus  in  the  initial  phase  of  model  development  was  to  construct  PC  pairs  that 
would  discriminate  across  a  variety  of  Navy  ratings. 

The  lists  of  community  areas,  work  styles,  work  environments,  and  work  activities 
were  then  judged  and  revised  by  SMEs  over  multiple  feedback  and  interview  sessions. 
Approximately  15  of  the  enlisted  community  managers  (ECMs),  who  manage  the 
enlisted  Navy  jobs,  served  as  the  key  group  of  experts.  At  the  outset  of  the  project  a  total 
of  45  work  activities  were  specified.  Through  numerous  discussions  it  was  determined 
that  a  reduction  in  the  number  of  job  tasks  was  warranted,  and  ultimately  19  activities 
were  eliminated.  These  efforts  resulted  in  a  list  of  8  community  areas,  4  work  styles,  4 
environments,  and  26  work  activities  (see  Appendix  A).  An  abbreviated  job  analytic 
procedure  was  utilized  to  link  specific  jobs  to  the  items  in  these  lists,  creating  a  79  by  41 
matrix.  The  SMEs  were  asked  to  indicate  each  pertinent  community  area,  work  style, 
environment,  and  work  activity,  in  the  ratings  they  manage.  Each  ECM  indicated 
roughly  five  activities  that  were  considered  to  be  important  or  critical  job  tasks 
performed  in  the  rating.  Ultimately,  these  data  formed  the  model  structure  that  linked 
the  79  individual  enlisted  Navy  jobs  with  each  item  under  the  4  work  dimensions  (see 
Appendix  B).  Interest  profiles  comprised  of  the  related  dimensional  qualities  (i.e., 
items)  could  then  be  created  for  each  job.  For  example,  job  analysis  data  revealed  that 
the  rating  Hospital  Corpsman  (HM)  should  be  linked  to  the  health  care  community, 
physical  work,  office  environment,  analyze  documents,  respond  to  emergencies,  serve 
customers,  etc. 
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Pictorial  I  tern  Development 

The  development  of  JOIN  items  was  an  iterative  process.  After  collecting  the  core 
content  for  the  interest  inventory,  the  next  phase  of  the  project  involved  the 
identification  of  images  that  were  representative  of  each  of  the  previously  identified 
work  dimensions.  Again,  the  decision  to  supplement  a  conventional  instrument  format 
(e.g.,  only  textual  descriptors)  with  digital  images  was  based  on  the  assumption  that 
applicants  have  little  prior  knowledge  about  the  range  of  tasks  that  are  performed  in 
Navy  jobs.  Based  on  this  premise,  it  was  thought  that  the  provision  of  pictures,  in 
conjunction  with  behavioral  descriptions  of  activities,  would  lend  itself  to  an  increased 
understanding  among  applicants  of  what  to  expect,  thereby  measuring  a  truer  level  of 
their  interest.  One  of  the  challenges  was  to  incorporate  pictures  of  enlisted  personnel 
performing  job  tasks  that  met  multiple  criteria.  In  order  to  be  selected  for  inclusion,  the 
digital  image  (l)  needed  to  be  representative  of  both  routine  and  non-routine  job  tasks, 
(2)  displayed  tasks  being  performed  in  various  community  areas,  as  well  as  in  assorted 
contextual  work  environments,  and  (3)  needed  to  display  both  gender  and  racial 
diversity. 

Various  resources  were  utilized  to  obtain  pictures  of  Navy  enlisted  personnel 
performing  various  work-related  tasks.  Specifically,  a  large  number  of  digital  images 
were  available  on  several  Navy  internet  websites.  In  addition,  there  are  several 
organizational  departments  within  the  Navy  (e.g.,  Navy  Recruiting  Command)  that 
regularly  photograph  the  personnel  out  in  the  fleet  and  provide  these  images  upon 
request.  The  main  objective  was  to  identity  and  categorize  photographs  into  an 
appropriate  work  activity  so  that  they  could  be  used  in  the  interest  inventory.  Over  500 
pictures  were  collected  that  were  considered  to  be  pictorially  representative  of  any  one 
of  the  previously  identified  activities  and  other  work  dimensions  (e.g.,  aviation,  outdoor 
environment,  etc.).  The  images  were  then  judged  by  two  groups  of  SMEs  during 
multiple  interview  sessions.  The  main  purpose  of  the  interview  sessions  was  to  validate 
the  content  of  the  collected  images.  The  goal  was  to  have  participants  categorize  each 
picture  and  at  the  same  time  identify  any  missing  work  activities  that  were  not 
suggested  during  previous  SME  interview  sessions.  Participants  were  shown,  in  random 
order,  a  numbered  photo  and  then  asked  to  identify  the  work  activity  that  the  picture 
best  represented.  If  the  picture  could  be  associated  with  more  than  one  activity,  the 
SMEs  were  asked  to  rank  order  their  choices  for  that  image.  The  multiple  feedback 
sessions  resulted  in  the  verification  of  the  more  than  300  images  now  contained  within 
JOIN. 

It  was  determined  that  each  time  a  work  dimension  item  was  presented  it  would  be 
accompanied  by  three  to  four  pictures  representing  that  item.  This  was  done  so  that 
various  examples  subsumed  under  an  individual  item  could  be  represented.  For 
example,  when  maintain  mechanical  equipment  was  presented  the  three  pictures 
displayed  would  contain  different  types  of  mechanical  equipment  that  ordinarily  would 
be  expressed  using  multiple  inventory  items.  Furthermore,  in  an  effort  to  obtain 
reliability  measures  on  the  pictorial  work  activity  items,  it  was  established  that  JOIN 
would  present  applicants  with  each  of  the  26  items  3  times  during  the  inventory  (i.e.,  78 
work  activity  items).  Based  on  applicant  feedback  and  additional  reliability  testing,  the 
number  of  presentations  to  applicants  was  reduced  from  three  to  one. 
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Computer-Administered  I  nventory  Development 

Throughout  the  content  development  and  image  identification  phase  of  the  project, 
the  researcher,  in  conjunction  with  computer  programmers,  focused  on  the 
development  of  a  computer-administered  vocational  interest  inventory.  The 
collaborative  effort  resulted  in  the  development  of  the  JOIN  experimental  software.  The 
JOIN  prototype  software  was  designed  to  provide  recruits  a  realistic  portrayal  of  the 
components  of  actual  Navy  ratings  (i.e.,  realistic  job  previews),  and  to  assess  their 
interest  in  four  broad  dimensions  of  work  associated  with  these  jobs.  This  was 
accomplished  by  employing  a  series  of  digital  images  of  Sailors  performing  job  tasks  in 
various  job  contexts. 

The  first  section  of  the  inventory,  Navy  community  areas,  consists  of  eight  areas  each 
represented  by  four  pictures  (see  Appendix  C).  Respondents  are  asked  to  indicate  their 
preference  (e.g.,  very  interested,  neutral,  not  interested)  using  a  5-point  Likert  scale. 

The  second  dimension  contains  four  items  identifying  different  work  styles  (e.g.,  work 
with  a  team),  and  the  third  dimension  contains  four  items  identifying  different  work 
environments  (e.g.,  work  outdoor).  Participants  are  asked  to  indicate  their  level  of 
preference  for  working  in  the  various  contextual  conditions,  each  represented  by  three 
pictures.  The  final  dimension,  work  activities,  includes  26  work  activity  items.  Each 
activity  serves  as  an  individual  interest  item  that  allows  participants  to  indicate  their 
level  of  interest  in  the  activity  presented  (e.g.,  maintain  mechanical  equipment). 
Participants  are  presented  with  three  pictures  and  descriptive  text  for  each  item,  and 
again  use  the  aforementioned  response  scale  (see  Appendix  D). 


Phase  11:1  nstrument  Testing 


Participants  and  Setting 

Data  were  collected  over  a  period  of  three  days,  during  which  time  three  to  four 
recruit  divisions  were  processed  per  day.  A  total  of  300  new  recruits  participated  in  the 
initial  testing  phase.  The  number  of  recruits  participating  in  usability  testing  per  day 
ranged  from  67  to  118  (day  3  and  day  2  respectively).  The  medical  and  dental  processing 
facility,  located  on  the  Naval  Training  Center  (NTC)  Great  Lakes  campus,  served  as  the 
test  site.  Participants  were  asked  to  voluntarily  participate  in  the  instrument’s  testing  as 
they  completed  a  number  of  processing  stages.  Random  selection  and  assignment  were 
not  feasible  due  to  the  imposition  of  constraints  related  to  limited  time  availability 
during  the  medical  and  dental  processing  phase.  Efforts  were  made,  however,  to  select  a 
representative  sample  of  minorities  (e.g.,  females  and  minority  races).  In  addition,  the 
participants  were  tested  on  either  their  third  or  fifth  day  of  basic  training  so  as  to  obtain 
a  sample  that  would  generalize  to  those  future  recruits  who  would  be  using  the 
instrument  once  made  operational  (i.e.,  naive  applicants  that  would  not  have  a  lot  of 
knowledge  regarding  the  Navy). 

Descriptive  statistics  indicated  that  45  enlisted  jobs  or  training  programs  (e.g., 
Hospital  Corpsman  [HM])  were  represented  within  the  sample  of  participants  (see 
Table  1).  Approximately  72  percent  of  respondents  were  male.  Respondents  ranged  in 
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age  from  17  to  32  years,  with  a  mean  age  of  19  years.  In  terms  of  race  or  ethnicity,  the 
sample  was  59  percent  Caucasian,  21  percent  African  American,  13  percent  Hispanic 
American,  and  2  percent  Asian  Pacific  Islander.  Approximately  7  percent  of  those 
surveyed  indicated  that  they  were  members  of  a  race  “other”  than  those  mentioned 
above.  The  proportion  of  female  and  other  minority  participants  was  found  to  be 
representative  of  the  current  Navy  population  of  enlisted  personnel  (U.S.  Navy,  2001). 


Table  1 

Recruit  sample  characteristics 


N 

% 

M 

SD 

Age 

290 

-- 

18.90 

2.11 

Gender 

290 

-- 

— 

-- 

Female 

82 

28.3 

— 

-- 

Male 

208 

71.7 

— 

— 

Ethnicity 

262 

-- 

— 

— 

Asian-Pacific  Islander 

4 

1.5 

— 

— 

African  American 

54 

20.6 

-- 

-- 

Caucasian 

154 

58.8 

— 

— 

Hispanic  American 

33 

12.6 

-- 

-- 

Other 

17 

6.5 

-- 

— 

Computer  Skills 

288 

-- 

3.89 

0.90 

1  (not  experienced) 

6 

2.1 

-- 

— 

2 

13 

4.5 

— 

— 

3 

58 

20.1 

— 

— 

4 

141 

49.0 

— 

— 

5  (very  experienced) 

70 

24.3 

-- 

— 

Computer  Hours 

270 

-- 

17.64 

22.86 

*  Percentages  reported  are  valid  percents. 


Procedure 

In  an  effort  to  provide  a  “first  look”  at  new  recruits’  opinions  of  the  instrument,  JOIN 
was  administered  to  a  group  of  naive  applicants  as  they  waited  in  the  medical  processing 
facility.  Participants  were  asked  to  voluntarily  participate  in  the  instrument  testing  as 
they  completed  a  number  of  processing  stages.  Sixteen  computer  stations  (i.e.,  a  table, 
laptop,  and  bench)  were  used  at  any  given  time,  with  five  to  six  participants  seated 
around  one  of  three  tables  set  up  in  a  large  processing  area.  Participants  were  asked  to 
complete  JOIN  and  provide  feedback  regarding  the  software.  Before  beginning  each 
session,  the  recruits  were  told  that  they  could  stop  at  any  time  and  their  participation 
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would  in  no  way  affect  their  current  or  future  Navy  career.  After  completing  JOIN 
feedback  data  were  collected  from  participants  in  two  ways.  All  participants  were  asked 
if  they  would  complete  an  optional  feedback  sheet  (see  Appendix  E).  In  addition, 
approximately  n  informal  discussion  groups  were  conducted  over  the  3-day  testing 
period.  The  discussion  groups  were  asked  many  of  the  same  questions  that  were 
included  on  the  feedback  sheet.  Additional  questions,  however,  were  added  daily  as 
needed  and  based  on  the  results  of  the  previous  test  day.  Completing  the  entire 
procedure  took  participants  an  average  of  40  minutes,  and  responding  to  only  the  JOIN 
inventory  an  average  of  24  minutes. 


Results 


Model  Analysis 

The  first  objective  was  to  design  a  model  that  would  distinguish  among  the  different 
Navy  enlisted  ratings.  The  data  collected  from  the  SMEs  during  the  developmental 
interview  sessions  were  analyzed  to  determine  the  uniqueness  of  the  PC  pairs  and  how 
well  they  cover  Navy  jobs.  Due  to  the  nature  of  the  SME  data,  a  principal  components 
analysis  was  employed  in  order  to  analyze  the  binary  data  that  linked  each  item  to  each 
job.  In  other  words,  the  procedure  determined  how  similar  or  dissimilar  particular  jobs 
were  from  one  another  based  on  the  inclusion  or  exclusion  of  a  particular  work  activity. 
A  principal  components  analysis  of  the  model’s  structure  was  conducted  and  indicated 
that  a  9-component  solution  with  varimax  rotation  provided  the  best  description  of  the 
data.  The  9-component  solution  accounted  for  89  percent  of  the  variance  among  the 
enlisted  jobs.  A  loading  of  0.40  was  the  criterion  established  for  interpreting  the  item 
composition  of  the  nine  components.  The  resulting  factors  that  were  extracted  seem  to 
fit  very  well  with  pre-established  Navy  occupational  groups.  This  initial  inquiry  into  the 
grouping  of  Navy  ratings,  based  upon  their  inherent  work  activity  structure  seems  to 
converge  with  previous  factor  analytic  results  conducted  on  Navy  jobs.  For  example,  the 
Navy  currently  categorizes  jobs  into  14  occupational  groups,  such  as  Administration, 
Mechanical,  Electrical,  Health  Care,  etc.  The  factor  analysis  revealed  that  the  enlisted 
ratings  were  grouped  in  a  similar  fashion  based  on  corresponding  items  (e.g.,  work 
activity).  The  components  extracted  were  interpreted  as  Electronic,  Mechanical, 
Administrative,  Mass  communication,  Construction,  Intelligence,  and  Weapons 
Activities. 

I  tem  Analyses 

The  next  objective  was  to  gather  responses  from  a  group  of  new  recruits  during  basic 
training  to  evaluate  the  psychometric  properties  of  the  pictorial  items  and  content  of  the 
computer-administered  instrument.  The  items  included  in  each  dimension  (i.e., 
community  areas,  work  styles-environments,  and  work  activities)  were  subjected  to 
various  analyses.  Descriptive  analyses  were  performed  on  all  items  to  determine  if  there 
is  adequate  variance  in  the  participants’  responses.  Furthermore,  assessment  of  the 
work  activity  items  took  place  on  the  individual  item  level  as  well  as  at  the  scale  level. 
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Community  Areas 


Participants  were  asked  to  rank  order  the  seven  Navy  community  areas  based  on 
level  of  interest.  While  respondents  were  required  to  indicate  at  least  one  area  they  were 
interested  in,  ranking  every  community  area  was  not  compulsory.  The  results  of 
descriptive  statistical  analyses  suggested  that  aviation  was  the  most  highly  sought  after 
community.  More  recruits  ranked  the  aviation  community  as  their  first  choice  than  any 
other  community  area  (33%).  Eighteen  percent  of  the  participants  indicated  that  they 
were  most  interested  in  special  programs,  whereas  the  health  care  and  intelligence 
communities  were  each  ranked  as  the  top  choice  by  14  percent  of  the  recruits  (i.e.,  14% 
and  14%,  respectively).  The  submarine  community  was  seemingly  least  preferred  among 
the  Navy  communities;  nearly  half  (49%)  of  the  participants  indicated  that  they  were 
not  interested  in  this  community.  Upon  examination,  the  results  indicated  that  the 
community  area  dimension  elicited  varied  responses  across  participants.  Specifically, 
with  the  exception  of  the  aviation  and  special  programs  community  areas,  the  other 
areas  appear  to  demonstrate  equal  dispersion  across  the  ranked  positions.  In  other 
words,  a  relatively  equal  number  of  participants  ranked  these  communities  as  their  first, 
second,  third,  fourth,  fifth,  sixth,  and  seventh  choices  (see  Table  2). 


Table  2 

Navy  community  area  item  response  distribution 


More 

1  nteresting 

Rank 

Less 

1  nteresting 

Not 

1  nteresting 

1 

2 

3 

4 

5 

6 

7 

(unranked) 

Aviation 

N 

94 

45 

39 

17 

13 

5 

4 

70 

% 

32.8 

15.7 

13.6 

5.9 

4.5 

1.7 

1.4 

24.4 

Construction 

N 

24 

26 

21 

17 

14 

22 

34 

129 

% 

8.4 

9.1 

7.3 

5.9 

4.9 

7.7 

11.8 

44.9 

Health  care 

N 

41 

27 

15 

18 

9 

24 

23 

130 

% 

14.3 

9.4 

5.2 

6.3 

3.1 

8.4 

8.0 

45.3 

Intelligence 

N 

43 

42 

40 

31 

19 

6 

6 

100 

% 

15.0 

14.6 

13.9 

10.8 

6.6 

2.1 

2.1 

34.8 

Submarine 

N 

15 

14 

31 

19 

25 

26 

16 

141 

% 

5.2 

4.9 

10.8 

6.6 

8.7 

9.1 

5.6 

49.1 

18 


Table  2 

Navy  community  area  item  response  distribution 


More 

1  nteresting 

Rank 

Less 

1  nteresting 

Not 

1  nteresting 

Surface 

N 

20 

40 

44 

45 

22 

15 

12 

89 

% 

7.0 

13.9 

15.3 

15.7 

7.7 

5.2 

4.2 

31.0 

Special 

Programs 

N 

51 

81 

39 

19 

19 

4 

4 

70 

% 

17.8 

28.2 

13.6 

6.6 

6.6 

1.4 

1.4 

24.4 

Special  Programs 

If  participants  indicated  an  interest  in  the  special  programs  community  area  they 
were  asked  to  respond  to  five  additional  items.  These  items  were  developed  to  assess  an 
interest  in  very  specialized  areas  that  have  additional  requirements  beyond  that  of  an 
enlisted  Navy  rating  alone.  The  five  programs  were  Sea/Air/Land  (SEAL),  Navy  Deep 
Sea  Diver  (DIVER),  Explosive  Ordnance  Disposal  (EOD),  Surface  Warfare  Combat 
Crewman  (SWCC),  and  Air  Search  and  Rescue  (AIRR).  The  participants  were  asked  if 
they  were  interested  in  four  very  specific  images  per  item,  and  they  were  asked  to 
respond  yes  or  no.  Two  hundred  and  seventeen  participants  indicated  an  interest  in 
Navy  special  programs.  Seventy-three  percent  of  these  recruits  indicated  that  they  were 
interested  in  being  a  Navy  SEAL.  A  relatively  equal  number  of  participants,  however, 
also  indicated  an  interest  in  the  other  four  programs  (see  Table  3). 


Table  3 

Participant  responses  for  the  Special  Program  Community 

area  items 


Special  Programs 

Not  interested 

1  nterested 

Air  Search  and  Rescue  (AIRR) 

N 

67 

150 

% 

30.9 

69.1 

Explosive  Ordnance  Disposal  (EOD) 

N 

77 

140 

% 

35.5 

64.5 

Navy  Deep  Sea  Diver  (DIVER) 

N 

100 

117 

% 

46.1 

53.9 
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Table  3 

Participant  responses  for  the  Special  Program  Community 

area  items 


Sea/Air/Land  (SEAL) 

N 

% 

59 

27.2 

158 

72.8 

Special  Warfare  Combat  Crewman 

(SWCC) 

N 

66 

150 

% 

30.6 

69.4 

Work-styles  and  Environments 

Participants  were  asked  to  indicate  their  interest  in  working  in  four  different  types  of 
styles  and  in  four  types  of  work  environments.  Participants  were  asked  to  indicate  their 
level  of  interest  in  these  work  dimensions  using  a  continuous  loo-point  scale,  with  o 
indicating  that  the  participant  was  not  interested,  and  too  indicating  very  interested. 
Analysis  of  work-style  preference  indicated  that  participants  preferred  working  with  a 
team  (M  =  77.49,  SD  =  25.59)  while  performing  physical  work  activities  (M  =  70.24,  SD 
=  28.02).  With  regard  to  a  work  environment,  participants  preferred  working  outdoors 
(M  =  79.55,  SD  =  27.31)  with  a  lower  preference  for  working  in  an  office  setting  (M  = 
49.01,  SD  =  35.38).  The  average  across  these  eight  items  (M  =  63.11)  and  the  large 
standard  deviations  indicate  that  participants’  interest  levels  vary  across  and  within 
these  work  dimensions  (see  Table  4). 


Table  4 

Work- Style  and  Work  Environment  item  descriptive  statistics 


M 

SD 

N 

Minimum 

Maximum 

Indoor 

55.17 

30.68 

287 

0 

100 

Outdoor 

79.55 

27.31 

287 

0 

100 

Office 

49.01 

35.38 

287 

0 

100 

Industrial 

44.74 

31.23 

287 

0 

100 

Mental 

62.75 

30.45 

287 

0 

100 

Physical 

70.24 

28.02 

287 

0 

100 

Work  1  ndependently 

65.93 

29.53 

287 

0 

100 

Work  with  a  Team 

77.49 

25.59 

287 

0 

100 

Analyses  were  performed  to  investigate  if  any  differences  existed  in  participants’ 
interest  in  different  work  styles  and  in  working  in  different  environments  when  gender 
and  race  were  used  as  grouping  variables.  Statistical  differences  were  found  when  each 
of  the  demographic  variables  was  considered.  Mean  differences  in  interest  level  existed 
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in  five  items  when  race  was  considered,  and  three  differences  were  found  when  gender 
was  considered.  It  seems  that  minority  races  indicated  a  larger  mean  interest  level  for 
working  independently,  t  (1,246)  =  2.65,  p  <  .05;  indoors,  t  (1,246)  =  2.34,  p  <  .05;  and 
in  an  office  environment,  t  (1,246)  =  4.04,  p  <.  05.  In  comparison,  non-minority 
participants  indicated  a  greater  preference  for  physical  work,  t  (1,246)  =  -2.11,  p  <  .05; 
in  an  outdoor  environment,  t  (1,246)  =  -2.42,  p  <  .05.  Finally,  results  for  this  dimension 
indicated  that  female  recruits  indicated  a  larger  mean  interest  level  for  work  in  an  office 
setting;  t  (1,273)  =  4-23>  P  <  -05;  whereas  males  indicated  a  greater  preference  for 
physical  work,  t  (1,273)  =  -2.70,  p  <  .05,  in  an  industrial  setting,  t  (1,273)  =  -2.34, 

P  <  -05. 

Work  Activity  I  tern  Analyses 


Descriptive  Statistical  Analyses 

Participants  were  asked  to  indicate  their  level  of  interest  regarding  26  work 
activities.  Each  work  activity  was  presented  in  a  random  order,  three  times,  with 
different  images  each  time,  and  again  a  100-point  response  scale  was  used.  An  analysis 
was  performed  to  determine  if  the  order  in  which  the  work  activity  items  were  presented 
influenced  the  level  of  interest  indicated.  A  general  trend  was  found  indicating  that  the 
later  in  the  inventory  an  individual  item  was  presented  the  lower  the  interest  level 
indicated.  With  regards  to  mean  interest  levels,  the  recruits  seemed  least  interested  in 
operating  facilities  and  most  interested  in  operating  weapons  (M  =  34.79,  SD  =  32.03 
and  M  =  73.94,  SD  =  31.24,  respectively).  Descriptive  statistics  for  each  of  the  work 
activity  items  are  presented  in  Table  5.  The  average  across  these  78  items  (M  =  48.28) 
and  the  large  standard  deviations  (ranging  from  29.10  to  35.20)  indicate  that 
participants’  interest  levels  vary  across  and  within  the  work  activities. 

Again,  with  these  items  additional  analyses  were  performed  to  investigate  if  any 
differences  existed  in  participants’  interests  when  gender  and  race  were  used  as 
grouping  variables.  Statistical  differences  were  found  when  these  demographic  variables 
were  considered.  Mean  differences  in  interest  level  existed  in  8  items  when  race  was 
considered,  and  16  differences  were  found  when  gender  was  considered.  Minority 
participants  indicated  a  larger  mean  interest  level  for  the  following  work  activities  when 
compared  to  non-minority  participants:  analyze  data,  t  (1,246)  =  2.21,  p  <  .05;  analyze 
documents,  1  ( 1,246)  =  2.20,  p  <  .05;  maintain  documents,  t  (1,246)  =  3.00,  p  <  .05; 
make  documents,  t  (1,246)  =  3.43,  p  <  .05;  operate  facilities,  t  (1,246)  =  3.10,  p  <  .05; 
operate  office  equipment,  t  (1,246)  =  4.54,  p  <  .05;  serve  customers,  t  (1,246)  =  2.59,  p  < 
.05;  and  train  people,  t  (1,246)  =  2.86,  p  <  .05.  Female  recruits  indicated  a  greater 
preference  for  the  following  activities:  analyze  documents,  t  (1,273)  =  3-84,  p  <  .05; 
maintain  documents,  t  (1,  273)  =  5.19,  p  <  .05;  make  communications,  t  (1,  273)  =  2.33, 
p  <  .05;  make  documents,  t  (1,  273)  =  3.85,  p  <  .05;  operate  facilities,  t  (1,  273)  =  2.85,  p 
<  .05;  operate  office  equipment,  t  (1,  273)  =  3.37,  p  <  .05;  and  serve  customers,  t  (1,  273) 
=  4-50,  p  <  .05.  In  comparison,  males  indicated  a  larger  mean  interest  level  in  the 
following  activities:  maintain  electrical  equipment,  1(1,273)  =  -4.33,  p  <  .05;  maintain 
electronic  equipment,  t  (1,273)  =  -4-66,  p  <  .05;  maintain  mechanical  equipment,  t 
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(1,273)  =  -4.43 ,p  <  .05;  maintain  weapons,  1  (1,273)  =  -7.38 ,p  <  .05;  make  mechanical 
equipment,  1  (1,273)  =  -4.01,  p  <  .05;  operate  electrical  equipment,  1  (1,273)  =  -3-38,  p  < 
.05;  operate  electronic  equipment,  1  (1,273)  =  -3.08,  p  <  .05;  operate  mechanical 
equipment,  1(1,273)  =  -4-3i,p  <  -05;  and  operate  weapons,  1(1,273)  =  -8.37,  p  <  .05. 


Table  5 

Work  Activity  item  descriptive  statistics  and  internal  consistency 

reliability  statistics 


a 

Item 

M 

SD 

Total 

0.9099 

-- 

-- 

-- 

Response  1 

48.23 

33.95 

Response  2 

48.39 

33.52 

Response  3 

48.23 

34.01 

Analyze  Communications 

0.8931 

Response  1 

44.44 

31.15 

Response  2 

45.31 

30.84 

Response  3 

44.36 

31.18 

Analyze  Data 

0.8706 

Response  1 

42.08 

32.20 

Response  2 

44.19 

31.42 

Response  3 

42.44 

32.07 

Analyze  Documents 

0.8613 

Response  1 

41.77 

31.67 

Response  2 

44.29 

31.25 

Response  3 

41.34 

32.32 

Direct  Aircraft 

0.9270 

Response  1 

54.29 

33.89 

Response  2 

52.21 

34.18 

Response  3 

53.61 

34.55 

Direct  Emergency  Response 

0.8974 

Response  1 

56.17 

32.68 

Response  2 

55.93 

32.66 

Response  3 

57.76 

32.84 

Maintain  Documents 

0.9068 

Response  1 

37.44 

31.79 

Response  2 

37.61 

31.24 

Response  3 

37.75 

31.57 

Maintain  Electrical  Equipment 

0.9357 

Response  1 

45.41 

33.50 

Response  2 

45.03 

33.38 

Response  3 

45.58 

33.60 

Maintain  Electronic  Equipment 

0.9128 

Response  1 

47.66 

33.56 

Response  2 

48.89 

33.52 

Response  3 

46.29 

33.24 
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Table  5 

Work  Activity  item  descriptive  statistics  and  internal  consistency 

reliability  statistics 


a 

Item 

M 

SD 

Maintain  Facilities 

0.8719 

Response  1 

34.45 

30.68 

Response  2 

34.29 

30.06 

Response  3 

34.71 

30.78 

Maintain  Mechanical  Equipment 

0.8687 

Response  1 

46.36 

33.18 

Response  2 

47.59 

32.60 

Response  3 

48.55 

34.32 

Maintain  Security 

0.8984 

Response  1 

61.84 

32.21 

Response  2 

61.06 

33.49 

Response  3 

56.91 

33.48 

Maintain  Supplies 

0.8842 

Response  1 

36.78 

29.82 

Response  2 

35.42 

29.10 

Response  3 

40.02 

30.99 

Maintain  Weapons 

0.9249 

Response  1 

58.68 

34.44 

Response  2 

61.05 

33.89 

Response  3 

61.89 

34.69 

Make  Communications 

0.8589 

Response  1 

45.41 

31.65 

Response  2 

44.68 

29.23 

Response  3 

45.70 

30.46 

Make  Documents 

0.8814 

Response  1 

38.03 

32.50 

Response  2 

38.22 

31.58 

Response  3 

39.27 

31.30 

Make  Facilities 

0.9452 

Response  1 

44.67 

34.34 

Response  2 

45.79 

34.43 

Response  3 

45.19 

34.86 

Make  Mechanical  Equipment 

0.9194 

Response  1 

46.17 

32.79 

Response  2 

44.32 

32.19 

Response  3 

44.01 

33.78 

Operate  Electrical  Equipment 

0.8980 

Response  1 

48.77 

32.03 

Response  2 

46.10 

32.74 

Response  3 

48.69 

32.65 

Operate  Electronic  Equipment 

0.8886 

Response  1 

52.37 

32.45 

Response  2 

51.25 

33.37 

Response  3 

54.12 

32.05 
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Table  5 

Work  Activity  item  descriptive  statistics  and  internal  consistency 

reliability  statistics 


a 

Item 

M 

SD 

Operate  Facilities 

0.9110 

Response  1 

35.09 

32.34 

Response  2 

37.53 

32.77 

Response  3 

31.73 

30.79 

Operate  Mechanical  Equipment 

0.8302 

Response  1 

56.47 

32.56 

Response  2 

53.24 

31.47 

Response  3 

52.55 

32.58 

Operate  Office  Equipment 

0.9342 

Response  1 

42.76 

35.20 

Response  2 

42.83 

33.65 

Response  3 

42.79 

33.66 

Operate  Weapons 

0.9185 

Response  1 

73.59 

32.21 

Response  2 

73.32 

30.84 

Response  3 

74.90 

30.74 

Respond  to  Emergencies 

0.9189 

Response  1 

63.66 

31.90 

Response  2 

64.40 

31.89 

Response  3 

67.05 

31.58 

Serve  Customers 

0.9073 

Response  1 

40.17 

34.33 

Response  2 

43.51 

34.39 

Response  3 

37.72 

33.38 

Train  People 

0.9053 

Response  1 

59.54 

32.07 

Response  2 

60.07 

30.96 

Response  3 

59.09 

32.18 

I  nternal  Consistency  Reliability  Analyses 

A  reliability  analysis  was  conducted  to  determine  the  internal  consistency  of  all  work 
activities.  In  addition  to  the  aforementioned  analysis,  the  internal  consistency  for  each 
individual  work  activity,  across  the  three  items  was  computed  (see  Table  5).  The  internal 
consistency  reliability  estimate  (or  alpha)  across  all  the  work  activities  was  very  good  (a 
=  0.91).  The  reliability  estimates  for  the  individual  work  activity  items  were  also  good, 
ranging  from  0.83  (operate  mechanical  equipment)  to  0.95  (make  facilities). 

Exploratory  Scale  Analyses 

An  exploratory  factor  analysis  of  participants’  responses  was  conducted  to  determine 
if  work  activity  items  could  be  grouped  into  interpretable  scales.  A  principal  axis  factor 
analysis  with  varimax  rotation  accounted  for  73  percent  of  the  variance  among  the  work 
activities,  and  indicated  that  a  10-component  solution  provided  the  best  description  of 
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the  data.  Each  of  the  10  factors  extracted  contained  work  activities  that  could  easily  be 
construed  as  the  individual  scales  (see  Table  6).  For  example,  all  of  the  work  activities 
describing  administrative  tasks  loaded  on  the  first  factor;  analyze  communications, 
analyze  data,  analyze  documents,  make  communications,  make  documents,  maintain 
documents,  and  operate  office  equipment.  Finally,  the  internal  consistency  of  the 
individual  aforementioned  scales  was  investigated.  The  reliability  estimates  were  very 
good,  ranging  from  0.96  for  the  administrative  duties  scale  to  0.90  for  the  security 
duties  scale. 


Table  6 

Factor  loadings  and  reliability  estimates  of  Work  Activity  item  scales 


Scale 

a 

Items 

Factor  Loading 

Administration 

0.96 

Analyze  Communications  1 

0.72 

Analyze  Communications  2 

0.69 

Analyze  Communications  3 

0.73 

Analyze  Data  1 

0.73 

Analyze  Data  2 

0.78 

Analyze  Data  3 

0.74 

Analyze  Documentsl 

0.76 

Analyze  Documents  2 

0.74 

Analyze  Documents  3 

0.72 

Make  Communications  1 

0.60 

Make  Communications  2 

0.64 

Make  Communications  3 

0.59 

Make  Documents  1 

0.65 

Make  Documents  2 

0.67 

Make  Documents  3 

0.66 

Maintain  Documents  1 

0.67 

Maintain  Documents  2 

0.71 

Maintain  Documents  3 

0.70 

Operate  Office  Equipment  1 

0.51 

Operate  Office  Equipment  2 

0.56 

Operate  Office  Equipment  3 

0.53 

Support  Service 

0.94 

Maintain  Facilities  1 

0.59 

Maintain  Facilities  2 

0.57 

Maintain  Facilities  3 

0.60 

Maintain  Supplies  1 

0.67 

Maintain  Supplies  2 

0.64 

Maintain  Supplies  3 

0.54 

Operate  Facilities  1 

0.81 

Operate  Facilities  2 

0.81 

Operate  Facilities  3 

0.79 
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Table  6 

Factor  loadings  and  reliability  estimates  of  Work  Activity  item  scales 


Support  Service 

Serve  Customers  1 

0.79 

Serve  Customers  2 

0.73 

Serve  Customers  3 

0.79 

Electrical  & 

0.96 

Maintain  Electrical  Equipment  1 

0.69 

Electronics 

Maintain  Electrical  Equipment  2 

0.70 

Maintain  Electrical  Equipment  3 

0.66 

Maintain  Electronic  Equipment  1 

0.77 

Maintain  Electronic  Equipment  2 

0.75 

Maintain  Electronic  Equipment  3 

0.76 

Operate  Electrical  Equipment  1 

0.70 

Operate  Electrical  Equipment  2 

0.73 

Operate  Electrical  Equipment  3 

0.77 

Operate  Electronic  Equipment  1 

0.73 

Operate  Electronic  Equipment  2 

0.66 

Operate  Electronic  Equipment  3 

0.63 

Mechanical 

0.94 

Make  Mechanical  Equipment  1 

0.73 

Make  Mechanical  Equipment  2 

0.76 

Make  Mechanical  Equipment  3 
Maintain  Mechanical  Equipment 

0.76 

1 

0.67 

Maintain  Mechanical  Equipment 

2 

0.65 

Maintain  Mechanical  Equipment 

3 

0.80 

Operate  Mechanical  Equipment 

1 

0.61 

Operate  Mechanical  Equipment 

2 

0.42 

Operate  Mechanical  Equipment 

3 

0.51 

Emergency 

Response 

0.93 

Direct  Emergency  Response  1 

0.77 

Direct  Emergency  Response  2 

0.75 

Direct  Emergency  Response  3 

0.73 

Respond  to  Emergencies  1 

0.82 

Respond  to  Emergencies  2 

0.87 

Respond  to  Emergencies  3 

0.78 

Ordnance 

0.93 

Maintain  Weapons  1 

0.70 

Maintain  Weapons  2 

0.67 

Maintain  Weapons  3 

0.72 

Operate  Weapons  1 

0.71 
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Table  6 

Factor  loadings  and  reliability  estimates  of  Work  Activity  item  scales 


Ordnance 

Operate  Weapons  2 

Operate  Weapons  3 

0.78 

0.75 

Direct  Aircraft 

0.93 

Direct  Aircraft  1 

0.77 

Direct  Aircraft  2 

0.74 

Direct  Aircraft  3 

0.79 

Construction 

0.95 

Make  Facilities  1 

0.73 

Make  Facilities  2 

0.69 

Make  Facilities  3 

0.74 

Training 

0.91 

Train  People  1 

0.78 

Train  People  2 

0.78 

Train  People  3 

0.72 

Law  Enforcement 

0.90 

Maintain  Security  1 

0.70 

Maintain  Security  2 

0.72 

Maintain  Security  3 

0.79 

Usability  Feedback 

One  final  objective  was  the  evaluation  of  the  usability  of  the  computer-administered 
instrument  using  recruits  to  determine  the  programs  functionality  under  certain 
organizational  constraints  (e.g.,  available  test  time,  recruit  characteristics).  Several 
questions  were  analyzed  on  the  feedback  sheet  designed  to  gauge  individual  differences 
in  level  of  computer  proficiency.  The  majority  of  participants  indicated  that  they  were 
experienced  computer  users,  spending  an  average  of  18  hours  a  week  before  basic 
training  using  a  computer.  In  addition  to  these  questions,  participants  were  asked 
various  questions  designed  to  assess  the  degree  to  which  JOIN  was  “user  friendly.”  The 
results  of  descriptive  statistical  analyses  suggested  that  JOIN  was  well  liked  by  the 
participants  (see  Table  7).  The  vast  majority  (89%)  of  recruits  indicated  that  JOIN  was 
easy  to  use.  Regarding  aesthetic  quality,  84  percent  of  recruits  indicated  that  they  found 
the  software  to  be  visually  appealing.  Participants  rated  the  instructions  (88%)  and  the 
tutorials  (76%)  as  being  either  good  or  very  good.  Fifty-six  percent  of  the  participants 
indicated  JOIN  was  very  good  compared  to  paper-and-pencil  questionnaires  that  they 
had  completed  in  the  past;  however,  only  16  percent  indicated  that  JOIN  was  very  good 
at  holding  their  attention  (32%  indicated  it  was  good).  This  latter  finding  was  attributed 
to  the  fact  that  participants  felt  that  JOIN  was  somewhat  repetitive,  as  indicated  in 
open-ended  questions  contained  on  the  feedback  sheet. 
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Table  7 

Feedback  item  response  statistics 


N 

% 

M 

SD 

Usability 

Ql.  How  would  you  rate  JOIN  on  the 
following: 

A.  Ease  of  use... 

300 

4.16 

0.71 

very  bad 

2 

0.7 

bad 

6 

2.0 

neither 

26 

8.7 

good 

174 

58.0 

very  good 

92 

30.7 

B.  Instructions... 

300 

4.15 

0.70 

very  bad 

1 

0.3 

bad 

5 

1.7 

neither 

34 

11.3 

good 

168 

56.0 

very  good 

92 

30.7 

C.  Tutorials... 

296 

3.93 

0.77 

very  bad 

3 

1.0 

bad 

6 

2.0 

neither 

63 

21.3 

good 

160 

54.1 

very  good 

64 

21.6 

D.  Visual  Appeal... 

298 

4.28 

0.87 

very  bad 

2 

0.7 

bad 

12 

4.0 

neither 

33 

11.1 

good 

103 

34.6 

very  good 

148 

49.7 

E.  Increasing  understanding  of  Navy  jobs... 

300 

3.65 

1.01 

very  bad 

10 

3.3 

bad 

24 

8.0 

neither 

93 

31.0 

good 

108 

36.0 

very  good 

65 

21.7 
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Table  7 

Feedback  item  response  statistics 


F.  Holding  your  attention... 

299 

3.32 

1.17 

very  bad 

26 

8.7 

bad 

45 

15.1 

neither 

83 

27.8 

good 

97 

32.4 

very  good 

48 

16.1 

Q2.  Compare  J  01 N  to  paper-and-pencil  tests 

298 

4.40 

.82 

very  bad 

4 

1.3 

bad 

6 

2.0 

neither 

23 

7.7 

good 

99 

33.2 

very  good 

166 

55.7 

Information  Processing 

Ql.  How  did  the  pictures  relate  to  the  main 
work  activity  headings 

291 

4.27 

.59 

very  bad 

0 

0.0 

bad 

2 

0.7 

neither 

16 

5.5 

good 

174 

59.8 

very  good 

99 

34.0 

Q2.  Your  understanding  of  what  each  picture 
represented  was 
very  bad 

287 

1 

.3 

4.25 

.69 

bad 

5 

1.7 

neither 

19 

6.6 

good 

157 

54.7 

very  good 

105 

36.6 

Q3.  How  did  you  make  your  decision  about 
your  level  of  interest  for  each  job  task? 

242 

2.92 

.99 

text 

17 

7.0 

pictures 

77 

31.8 

both  text  and  pictures 

56 

23.1 

other 

92 

38.0 

If  you  like  one  picture  "very  much"  ...how 
would  you  respond  on  the  scale  below? 

284 

2.81 

.52 

below 

16 

5.6 

halfway 

21 

7.4 

above 

247 

87.0 
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The  data  from  several  of  the  feedback  questions  were  intended  to  elucidate  the 
degree  to  which  the  textual  descriptions  and  pictures  contained  in  JOIN  influenced 
participants’  level  of  interest.  The  majority  of  recruits  (94%)  indicated  that  main  work 
activity  headings  (e.g.,  Analyze  Communications  -  Receive  and  interpret  visual,  audio, 
verbal,  and  written  communication)  were  related  to  the  pictures  (i.e.,  the  pictorial 
representation  of  the  text).  Furthermore,  91  percent  of  the  participants  reported  that 
they  had  a  good  understanding  of  what  the  pictures  represented.  Thirty-one  percent  of 
the  participants  indicated  that  they  used  the  pictures  to  make  their  decision  regarding 
their  level  of  interest  in  each  work  activity,  while  23  percent  reported  that  they  used 
both  the  text  and  the  pictures. 

Data  from  the  responses  to  the  open-ended  feedback  items  were  also  examined. 
Overall,  the  participants  liked  the  computer-adapted  format  and  the  pictures  contained 
within  JOIN.  The  most  often  cited  dislike  stemmed  from  the  perceived  redundancy  of 
items.  Each  work  activity  was  presented  three  times  in  order  to  assess  internal 
consistency.  Also,  many  of  the  items  contained  either  a  similar  process  (i.e.,  analyze, 
maintain,  operate,  etc.)  or  a  similar  content  (e.g.,  mechanical  equipment, 
communications,  electrical  equipment,  etc.).  These  two  points  may  explain  the 
consistent  reporting  of  the  redundancy  and  repetitiveness  of  JOIN. 

Discussion  Groups 

A  total  of  57  recruits  participated  in  11  discussion  groups,  4  of  which  had  all  female 
participants  (N  =  21).  Six  groups  were  conducted  on  the  first  day,  three  on  the  second, 
and  two  on  the  final  day  of  testing.  Fewer  discussion  groups  were  conducted  on  the 
second  and  third  days  of  testing,  as  the  duplication  of  the  information  generated  within 
group  sessions  conducted  on  the  first  day  of  testing  was  readily  apparent  (e.g., 
saturation  was  occurring).  Indeed,  there  was  very  little  variation  in  the  responses  from 
the  participants  on  the  first  day  to  the  last  day.  These  groups  were  used  as  a  platform  to 
discuss  emerging  questions  or  problems  that  had  been  identified  on  a  day-to-day  basis 
during  testing.  For  all  intents  and  purposes,  the  information  obtained  from  these  groups 
coincided  with  the  feedback  data  reported  earlier. 


Discussion 


This  research  project  was  a  necessary  first  step  to  the  inclusion  of  an  interest 
inventory  in  the  current  Navy  classification  system.  Currently  there  is  not  an  instrument 
available  that  meets  the  specific  requirements  of  the  Navy  that  would  enable  the 
collection  of  applicant  job  interests,  in  a  very  limited  amount  of  time,  and  incorporate 
those  responses  in  a  classification  decision.  The  main  goal  of  the  proposed  research 
project  was  to  develop  an  instrument  that  would  assess  the  specific  work  interests  of  a 
population  of  applicants  that  have  little  prior  knowledge  of  the  Navy’s  world  of  work. 
Several  hypotheses  were  tested  during  the  investigation. 
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The  initial  results  from  the  usability  testing  presented  in  this  paper  are  very 
promising  on  several  levels.  The  first  aim  was  accomplished  and  pictorial  items  for  an 
interest  instrument  were  developed  that  represent  various  Navy  work  areas,  work  styles, 
work  environments,  and  work  activities.  The  second  objective  was  to  gather  responses 
from  a  group  of  new  recruits  during  basic  training  to  evaluate  the  psychometric 
properties  of  the  pictorial  items  and  content  of  the  computer-administered  instrument. 
The  initial  evaluation  regarding  the  item  content  of  the  instrument  included  descriptive, 
reliability,  and  factor  analyses.  The  descriptive  statistical  analyses  of  the  JOIN  items 
indicated  that  there  is  adequate  dispersion  of  individual  responses.  In  other  words,  the 
participants  were  different  in  their  level  of  interest  in  various  items.  Also,  the  statistical 
reliability  of  the  work  activity  items  was  assessed  and  the  developed  items  appear  to  be 
very  consistent  in  measuring  participant  interest  in  the  individual  enlisted  rating  job 
tasks. 

The  final  objective  was  the  evaluation  of  the  usability  of  the  computer-administered 
instrument  with  the  recruits  to  determine  the  program’s  functionality  under  certain 
organizational  constraints  (e.g.,  available  test  time,  recruit  characteristics).  The  results 
from  the  analysis  of  feedback  items  and  discussion  sessions  suggest  that  incoming 
recruits  possess  the  level  of  computer  proficiency  necessary  to  complete  the  inventory. 
Finally,  with  the  exception  of  comments  related  to  the  repetition  of  items,  recruits  were 
positive  in  their  evaluation  of  the  quality  of  the  computer-adapted  inventory.  Initial 
results  suggest  that,  recruits  found  the  JOIN  software  to  be  a  user-friendly  interface. 

By  design,  the  future  success  of  JOIN  is  predicated  on  its  predictive  utility.  That  is, 
the  success  of  JOIN  is  contingent  upon  the  degree  to  which  the  consideration  of 
interests  leads  to  better  recruit-rating  (i.e.,  recruit-job)  classification  decisions.  Future 
testing  will  allow  for  the  assessment  of  the  extent  to  which  interests  account  for  unique 
variance  over  and  above  preexisting  screening  measures  (e.g.,  mental  aptitude,  medical 
conditions,  etc.).  If  subsequent  testing  and  analyses  suggest  that  the  consideration  of 
interests  leads  to  better  recruit-rating  (i.e.,  recruit-job)  classification  decisions,  then  we 
may  conclude  that  the  administration  of  JOIN  to  new  recruits  is  a  worthwhile  endeavor. 

The  JOIN  prototype  software  is  in  its  initial  phase  of  development,  and,  as  such,  the 
functionality  of  subsequent  versions  is  potentially  limitless;  JOIN  need  not  be  relegated 
to  measuring  interests  alone.  Although  there  are  numerous  potential  applications  for 
the  JOIN  software,  one  prospect  is  of  particular  interest.  Specifically,  in  addition  to 
determining  what  Navy  ratings  prospective  recruits  are  interested  in,  JOIN  could 
actually  function  to  “sell”  the  Navy.  Should  JOIN  function  in  this  capacity  in  the  future, 
it  would  necessitate  an  in-depth  examination  of  the  salient  choice  criteria  that  underlie 
decisions  with  regards  to  competing  organization/job  alternatives.  Such  considerations 
could  be  addressed  as  subsequent  versions  of  JOIN  are  released. 
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Appendix  A: 

J  obs  and  Occupational  I  nterest  in  the  Navy  (J  Ol  N) 

Textual  Items 
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Table  A-l 

J  ob  Opportunities  in  the  Navy  (J  Ol  N)  Textual  I  terns 


Navy  Community  Areas 

Aviation 

Submarine 

Construction 

Surface 

Health  Care 

Special  Programs 

Intelligence 

Support 

Work  Styles 

Mental 

Work  independently 

Physical 

Work  with  a  team 

Work  Environments 

Indoor 

1  ndustrial 

Outdoor 

Office 

Work  Activity  1  terns 

Analyze  Communications— Receive  and  interpret  visual,  audio,  verbal,  and  written 
communication 

Analyze  Data— Compile  and  interpret  visual,  audio,  verbal,  and  written  information 
Analyze  Documents— Interpret  written,  photographed,  and  illustrated  material 
Direct  Aircraft— Organize,  dispatch,  and  guide  airplanes  and  helicopters 
Direct  Emergency  Response— Organize,  dispatch,  and  guide  personnel  to  react  to 
critical  and  urgent  situations 

Maintain  Documents— Organize,  review,  and  update  written,  photographed,  and 
illustrated  material 

Maintain  Electrical  Equipment— Service  and  repair  motors,  generators,  and  switchboards 
Maintain  Electronic  Equipment— Service  and  repair  sonar,  radar,  communication,  and 
system  monitoring  equipment 
Maintain  Facilities— Sustain  an  area  of  operation 

Maintain  Mechanical  Equipment— Monitor,  service,  and  repair  machinery  and  simple 
mechanical  equipment 

Maintain  Security— Monitor,  guard  and  police  Navy  ships,  aircraft,  and  shore 
installations 

Maintain  Supplies— Monitor  and  sustain  inventory 

Maintain  Weapons— Service  and  repair  small  arms,  missile  systems,  and  other 
munitions 

Make  Communications— Create  and  communicate  visual,  audio,  verbal,  and  written 
information 

Make  Documents— Produce  written,  photographed,  and  illustrated  material 
Make  Facilities— Construct  buildings,  bridges,  and  walls 
Make  Mechanical  Equipment— Manufacture  and  fabricate  machinery  and  simple 
mechanical  equipment 

Operate  Electrical  Equipment— Activate,  drive,  and  control  motors,  generators,  and 
switchboards 
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Table  A- 1 

J  ob  Opportunities  in  the  Navy  (J  Ol  N)  Textual  I  terns 


Operate  Electronic  Equipment— Operate  sonar,  radar,  communication,  and  system 
monitoring  equipment 

Operate  Facilities— Manage,  regulate,  and  coordinate  an  area  of  operation  (e.g.,  barber 
shops,  laundry,  ship  store,  vending,  retail) 

Operate  Mechanical  Equipment— Activate,  drive,  and  control  machinery  and  simple 
mechanical  equipment 

Operate  Office  Equipment— Use  standard  office  related  equipment  such  as  typewriters, 
adding  machines,  cameras,  presses,  file  cabinets,  etc. 

Operate  Weapons— Shoot,  fire,  and  control  small  arms,  missile  systems,  and  other 
munitions 

Respond  to  Emergencies— React  to  and  take  appropriate  measures  in  critical  and  urgent 
situations 

Serve  Customers— Provide  goods  and  services  to  patrons  and  consumers 

Train  People— I  nstruct  and  develop  personnel  in  professions,  skills,  or  occupations 


Appendix  B: 
J  Ol  N  Model 
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Navy  Rating  by  Item  Matrix 


Community  Areas... 


Job 

Aviation 

Construction 

Healthcare 

Surface 

Yeoman,  YN 

1 

1 

0 

1 

Hospital  Corpsman,  HM 

0 

0 

1 

0 

Builder,  BU 

0 

1 

0 

0 

Air  Traffic  Controller,  AC 

1 

0 

0 

1 

Work  Styles... 

Work  Environments... 

Job 

Physical 

Work  1  ndependently 

Office 

Outdoor 

YN 

0 

1 

1 

0 

HM 

0 

0 

1 

0 

BU 

1 

0 

0 

1 

AC 

0 

1 

1 

0 

Work  Activities... 


Job 

Analyze 

Data 

Direct 

Aircraft 

Maintain 

Documents 

Make 

Facilities 

Operate 

Weapons 

Serve 

Customers 

Train 

People 

YN 

0 

0 

1 

0 

0 

1 

0 

HM 

0 

0 

1 

0 

0 

1 

0 

BU 

0 

0 

0 

1 

1 

0 

0 

AC 

1 

1 

1 

0 

0 

0 

0 

*Note:  A  79  x  41matrix  was  established  with  the  Navy  jobs  listed  in  each  row  and  the  dimensions  of  work  were  in 

each  column.  A  "1"  in  the  cell  indicates  the  item  is  relevant  to  that  job,  and  a  "0"  indicates  the  item  is  absent 
from  that  particular  job. 
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Appendix  C: 

J  Ol  N  Community  Areas  Sample  Screen 
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J  obs  Opportunities  in  the  Navy  (J  Ol  N) 
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Appendix  D: 

J  Ol  N  Work  Activity  Sample  Screen 
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Appendix  E: 

J  Ol  N  Feedback  Data  Sheet 
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J  Ol  N  Feedback 


The  purpose  of  this  questionnaire  is  to  evaluate  JOIN.  Your  responses  will  be 
analyzed  and  maintained  by  NPRST  (PERS-13).  All  responses  will  be  held  in  confidence. 
Information  provided  will  be  summarized  and  will  not  be  attributable  to  individuals. 
Your  participation  is  completely  voluntary.  Failure  to  respond  will  NOT  result  in  any 
penalties  with  the  exception  of  lack  of  representation  of  your  views  in  final  results  and 
outcomes. 

Instructions:  Please  take  a  moment  to  reflect  back  on  JOIN  and  answer 
the  following  questions. 


Usability 

1.  How  would  you  rate  JOIN  on  the  following: 

a.  Ease  of  use... 

Explain 

b.  Instructions... 

Explain 

c.  Tutorials... 

Explain 

d.  Visual  appeal... 

Explain 

e.  Increasing  your  understanding  of  Navy  jobs... 

Explain 

f.  Holding  your  attention... 

Explain 

2.  Compared  to  paper  and  pencil  questionnaires  I  have  taken,  the  computerized 
version  of  JOIN  was... 

Explain 
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Instructions:  Please  take  a  moment  to  reflect  back  on  the  pictures  used 
in  JOIN  and  answer  the  following  questions. 

I  nformation  Processing 

3.  How  did  the  pictures  relate  to  the  main  work  activity  headings  (e.g.,  Analyze 
data)? 

4.  Your  understanding  of  what  each  picture  represented  was... 

5.  How  did  you  make  your  decision  about  your  level  of  interest  for  each  job  task? 

6.  If  you  liked  one  picture  “very  much”  out  of  the  three  presented,  how  would  you 
respond  on  the  scale  below?  Mark  an  “X”  on  the  scale  to  indicate  your  response. 

*Note:  A  5-point  response  scale,  ranging  from  very  bad  to  very  good  and 
including  neither,  was  used  for  question  la-if,  2,  3,  and  4. 

Instructions:  Please  answer  the  following  questions  completely. 

Feedback 

1.  What  did  you  like  about  JOIN? 

2.  What  did  you  dislike  about  JOIN? 

3.  Suggestions/  Recommendations? 

Background  I  nformation 

1.  With  regard  to  your  computer  skills,  would  you  say  that  you  are...  (5-point  scale, 
not  experienced  to  very  experienced) 

2.  How  many  hours  per  week  do  you  spend  using  a  computer? 

3.  What  is  your  rating,  or  what  training  program  are  you  currently  in?  or  mark  if 
undesignated 

4.  Age 

5.  Gender 

6.  Ethnicity 
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Distribution 


AIR  UNIVERSITY  LIBRARY 

ARMY  MANAGEMENT  STAFF  COLLEGE  LIBRARY 

ARMY  RESEARCH  INSTITUTE  LIBRARY 

ARMY  WAR  COLLEGE  LIBRARY 

CENTER  FOR  NAVAL  ANALYSES  LIBRARY 

DEFENSE  TECHNICAL  INFORMATION  CENTER 

HUMAN  RESOURCES  DIRECTORATE  TECHNICAL  LIBRARY 

JOINT  FORCES  STAFF  COLLEGE  LIBRARY 

MARINE  CORPS  UNIVERSITY  LIBRARIES 

NATIONAL  DEFENSE  UNIVERSITY  LIBRARY 

NAVAL  HEALTH  RESEARCH  CENTER  WILKINS  BIOMEDICAL  LIBRARY 
NAVAL  POSTGRADUATE  SCHOOL  DUDLEY  KNOX  LIBRARY 
NAVAL  RESEARCH  LABORATORY  RUTH  HOOKER  RESEARCH  LIBRARY 
NAVAL  WAR  COLLEGE  LIBRARY 

NAVY  PERSONNEL  RESEARCH,  STUDIES,  AND  TECHNOLOGY  SPISHOCK 
LIBRARY  (3) 

PENTAGON  LIBRARY 

USAF  ACADEMY  LIBRARY 

US  COAST  GUARD  ACADEMY  LIBRARY 

US  MERCHANT  MARINE  ACADEMY  BLAND  LIBRARY 

US  MILITARY  ACADEMY  AT  WEST  POINT  LIBRARY 

US  NAVAL  ACADEMY  NIMITZ  LIBRARY 


